{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0f01763d-99dd-5b46-bafb-2bf45a5bf9e6","task_key":"default--validation--35685","task_revision_id":"1","upstream_id":"35685","short_description":"HellaSwag validation 35685","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to understand what is being asked of you in an essay or exam [title] Read all of the directions. [step] No matter how many wrong answers you have, as long as you follow all the directions, you will get some credit. Re-read the question slowly.\",\"endings\":\"[\\\"Read it again and see how you can change the answer. If it doesn't do anything helpful, you will not have any credit at all.\\\", \\\"[substeps] If you question an idea's answer multiple times, you can easily become stumped and you will not have a lot of time to reshape your answer. Instead, rework the idea of a question by reviewing the given reading.\\\", 'Once you are finished, review everything you have read. Look for errors, change the question box, or rearrange the wording to be just right.', 'In your haste and panic, you may have missed the point. Then, analyze what you have learned and start writing.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}