{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"28e65a44-b547-58a0-81bc-39a66692e445","task_key":"default--validation--33813","task_revision_id":"1","upstream_id":"33813","short_description":"HellaSwag validation 33813","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to measure for jeans [title] Put on the shoes you plan to wear with your jeans while taking the measurements. [step] You will finish taking the measurements at approximately where your shoes will meet the cuff. If you wear any orthotic inserts in your shoe, put them on as well to ensure an accurate reading.\",\"endings\":\"['[title] Stand with your back against a wall. [step] Keep your back as straight as possible so you can get precise measurements.', '[title] Pull the cuff down and over your wrist. [step] You will be placing your hands near the original measurement which you took in the previous step.', '[substeps] Push your right foot back against the lead edge of the shoe and use your left hand to point the toe of the shoe towards your foot. Bring the shoe inward so that your ankle is on the shoe and your right heel is on the other side.', '[title] Make an x in the fabric of your cloth shoe. [step] Let the fabric of your cloth shoe absorb the warm of the foot so that the inside of your shoe will get the hottest.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}