{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bb411c7d-c9e7-56bd-b77c-6e9d69c79246","task_key":"default--validation--22996","task_revision_id":"1","upstream_id":"22996","short_description":"HellaSwag validation 22996","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wear an infinity scarf [title] Place the scarf behind your neck. [step] Grab the ends of the scarf so that the scarf is stretched horizontally behind your neck. The scarf should be parallel to your shoulders.\",\"endings\":\"['Tug the scarf straight across on both sides, keeping your shoulders back, working in an x shape. [substeps] Your goal here is to make the scarf fully taut-for better protection, round the waist, etc.', '[substeps] The entire scarf should be behind your neck. Do not insert your head through the loop of the infinity scarf.', 'Spread your fingers over the scarf so that they are close together. Place one hand on the left end of the scarf and one hand on the right.', '[substeps] The ends will be bent so that the ends are stacked vertically together. [title] Line up your shoulders, taking care to move around as much as possible.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}