{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"89e5dc20-b6bc-551d-b566-811b1f78e5d8","task_key":"default--validation--49284","task_revision_id":"1","upstream_id":"49284","short_description":"HellaSwag validation 49284","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to lace timberlands [title] Thread one side of the lace through the bottom eyelet. [step] The eyelets are the small holes on the front of the boot. You can thread the lace through the eyelet from the outside or push the laces in through the holes from the middle of the shoe.\",\"endings\":\"['[substeps] Make sure you thread the lace through both eyelets. [title] Loop the lace through both eyelets on the front and sides of the boot.', '[title] Hook the second end of the lace between two eyelet holes on the inside of the boot. [step] Pass the lace through one eyelet on the inside and then slip the other end of the lace through to the other eyelet hole.', 'Try both methods to see which style you prefer. [substeps] Threading the laces from the outside lends itself well to the loose and criss-cross styles.', \\\"[title] Thread your lace through the eyelet hole. [step] As you're pulling the lace up to the eyelet hole, shift the lace up and across the shoe so that the lace is exposed to the eyelet hole.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}