{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"df98023b-58e8-5e68-a5e1-1205116fd087","task_key":"default--validation--35707","task_revision_id":"1","upstream_id":"35707","short_description":"HellaSwag validation 35707","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to adopt a russian baby [title] Fill out form i-600a, application for advance processing of orphan petition. [step] This form is available through the u.s. citizenship and immigration services, and it is the necessary first step in obtaining an immigrant visa for your adoptive child. You do not need to have a specific child in mind to complete this form.\",\"endings\":\"['[substeps] The form needs to include information about the child and the country they are from. Don\\\\'t forget to include all the information that you know: \\\" parent, child, next of kin.', '[title] Contact the uscis or a state-licensed adoption agency to request a home study. [step] Home studies must be performed in order to determine the fitness of you and the home environment you intend to bring a child into.', '[title] Complete section ii of form ii of visa application, application for advance processing of a russian baby. [step] This form is available through the u.s.', 'The form is brief to provide basic information about yourself and your adoptive child, as well as what you are looking for and need to bring with you. [title] Submit the completed form via the mail or by fax.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}