{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"842c4579-ddda-519e-8796-a3fc613a7da6","task_key":"default--validation--24129","task_revision_id":"1","upstream_id":"24129","short_description":"HellaSwag validation 24129","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to eat food on a banana leaf [title] Forego utensils. [step] Use your hand to eat most dishes. Enhance your dining experience with all five senses, including touch.\",\"endings\":\"['Put a fork-or a spoon, left-handed or right-handed-on the table. [title] Wash your hands with warm, soapy water before and after eating.', 'This is particularly important during breaks, when utensils become unattractive; nutrition experts suggest eating no more than three or four food items a day in order to assist weight loss in lowering blood pressure. Stick to foods that are healthy and provide all four senses.', 'Things to keep in mind during the meal : [substeps] Taste. Aroma is feeling, smell is appearance, taste is all.', 'Enjoy the added benefit of never burning your mouth or tongue, since any food that is too hot to touch will have to cool off before it reaches your mouth. [substeps] Serving utensils are often used to collect food from a communal platter.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}