{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"082e5cab-a762-507f-b19a-1e302b53a53d","task_key":"default--validation--24943","task_revision_id":"1","upstream_id":"24943","short_description":"HellaSwag validation 24943","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to make a sterile bacterial smear [title] Remove all unneeded equipment and objects from the lab bench. [substeps] Leaving objects on the lab bench can get in the way or be a safety hazard. Paper left on the bench can catch fire if a bunsen burner is knocked over.\",\"endings\":\"['[title] Squirt liquid disinfectant onto the lab bench. [substeps] Use a liquid disinfectant meant for hard surfaces, containing bleach or ethanol.', 'Leaving objects off your bench is the key to your success. Removing one item from the lab bench is not recommended unless you want to collect germs.', 'Put any food or also any cutlery or other items that may have been sitting on the bench away from where the patient is working. Sanitize hands, feet, clothing, and anything else in the area that would be touched by the patient.', 'Certain equipment, like gloves, will drop out of the solution and become a fire hazard and potentially lead to serious illness and death. Important objects to remove include any infected equipment you may have, poultry or other food items such as fruit or vegetables.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}