{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"da6396ff-1ba6-575f-b907-ab16d0e8f97d","task_key":"default--validation--29872","task_revision_id":"1","upstream_id":"29872","short_description":"HellaSwag validation 29872","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to clean cuts [title] Wash your hands. [step] Before attempting to clean a cut, either on yourself or on another person, it is key that your own hands be clean. This way, you will not introduce any germs or foreign substances into the cut that may later cause it to become infected.\",\"endings\":\"['[substeps] If available, put on some single-use gloves after washing your hands for more protection [title] Stop the bleeding. [step] Before cleaning the cut, it is also key to stop the bleeding.', 'You must wash your hands thoroughly after you touch any part of your skin that is touched. [substeps] To clean your cuts, wash them thoroughly in warm water and soap before and after touching your cuts.', 'The simplest solution for a cut that is on your hand is to wash it with soap and warm water to begin your cleaning process. [substeps] To wash the cut, first treat it with soap and water.', '[substeps] If you wear rubber or gloves, wash those first with warm water. If you wear latex gloves, wash those too.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}