{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ceb5af84-eac1-5bec-a94b-906f959d3859","task_key":"default--validation--13679","task_revision_id":"1","upstream_id":"13679","short_description":"HellaSwag validation 13679","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to elope [title] Elope at a courthouse for the simplest route. [step] Choosing to elope by registering with the government is not only the most straightforward way but also best if you want to avoid all the fuss of weddings. If you want, you can make it just between the two of you.\",\"endings\":\"[\\\"The important thing is that you and your partner (or parents) can agree that you want to get married as a couple. [substeps] If one of you is opposed to the idea, proceed straight to officiant's office as soon as possible.\\\", \\\"Plus, you won't have to fuss around with all of the legal documents. [substeps] Make sure you bring all necessary documents with you: take government-issued ids, birth certificates, social security cards, and (depending on your country) blood test results.\\\", '[substeps] The courthouse in texas is located in the executive district of texas. Students and families who plan to marry at court should also register to go to a courthouse in texas.', '[substeps] You can reserve an annulment in the event of a pending lawsuit. Choose a county where the judge approves an annulment, such as montana or arkansas.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}