{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"da732d5b-9556-5f7d-9435-d414b8c902b2","task_key":"default--validation--45105--variant-5596fd56652c","task_revision_id":"1","upstream_id":"45105--variant-5596fd56652c","short_description":"HellaSwag validation 45105","config":"default","split":"validation","body":"{\"activity_label\":\"Work World\",\"ctx\":\"[header] How to become an officer in the military [title] Understand service academies. [step] The military offers undergraduate educational opportunities to the most determined and hard working high school students. Most programs offer a large grant or scholarship to the accepted students.\",\"endings\":\"['If your school has an annual grant, applicants can choose to transfer it to all major institutions. Of course, they can also apply to international colleges or universities, though these schools are not always the most prestigious.', 'An increasingly much increased number of single-year mp students are required to work in the military. Depending on age, many mp students are required to work in military academies and eventually retire into the elite military school of their choice.', 'The most preferred course for college students is basic physics. [substeps] Keep in mind that the requirements vary between schools.', \\\"Service academy offers full four year scholarship and require five years commitment to the service after graduation. [substeps] You'll graduate with a bachelor of science's degree and enter the force as either a second lieutenant or ensign.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}