{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"13eacf1a-2d58-5873-86f6-e1376ed71c1d","task_key":"default--validation--9238","task_revision_id":"1","upstream_id":"9238","short_description":"HellaSwag validation 9238","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to use a broiler [title] Find the broiler. [step] Older gas ovens have a drawer at the bottom of the oven that contains the broiling unit. This compartment is commonly called the \\\" broiler drawer.\",\"endings\":\"['\\\" broilers have a smaller compartment in the oven, and this is also called a \\\" broiler drawer. \\\" [substeps] When you\\\\'re using a stove top broiler, the drawer is easy to find.', '\\\" if you cannot find a broiler drawer, you can also find in your refrigerator. The broiler drawer also contains a knob for the broiler unit.', '\\\" if your oven doesn\\\\'t have a broiler drawer, the broiler is inside the main oven compartment. In this case, the broiling unit in will be attached to the top of the oven interior.', '\\\" some broilers have compartment f. Check the pan drawer, as older gas ovens have compartment 6.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}