{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c08c9f75-55f4-5928-ab71-73a607a9b8e1","task_key":"default--validation--8430","task_revision_id":"1","upstream_id":"8430","short_description":"HellaSwag validation 8430","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to get married in wisconsin [title] Check if you can get married. [step] Wisconsin has a few qualifications that you must meet. Check ahead of time : [substeps] You must be at least 18 years old to get married.\",\"endings\":\"['You must have been born in wisconsin and have certain education or firearm titles, if any. You must have supervised government forces that help you, such as military bases.', \\\"If you are 16 or 17, you'll need both parents' consent or the consent of your legal guardian. There is a consent form available at the county clerk's office.\\\", 'Under the 180º\\\" your state \\\" act, the recipient is a legal resident of wisconsin. In which state you live, you are legally legally unmarried.', 'You must meet the same age as you are before you can get married in wisconsin. You will need to be a resident of wisconsin for at least six months and the applicant must be a u.s.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}