{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8f9419c0-aafc-5eea-a2c2-3dceaa86e74b","task_key":"default--validation--46568","task_revision_id":"1","upstream_id":"46568","short_description":"HellaSwag validation 46568","config":"default","split":"validation","body":"{\"activity_label\":\"Cars & Other Vehicles\",\"ctx\":\"[header] How to become a flight crew scheduler [title] Graduate from high school. [substeps] Airlines, including small, commuter airlines will not hire you if you do not have a high school diploma. [title] Learn how to use a computer.\",\"endings\":\"['[substeps] Before you begin working in a field that requires a computer, learn how to use a computer. Most websites have some information about how to use a computer.', '[step] Go to university and put your paper-grade grades at the bottom of your \\\" to-do \\\" list so that you have access to an available computer. Research the best computer operating systems in the country.', '[substeps] To become a flight crew scheduler, you must have at least the basic computer skills of your own. In both state and federal aviation, it is illegal to inflate mechanical skills with physical gifts as they are prohibited by airline regulations.', '[substeps] Flight crew schedulers spend the majority of their time at work on computers. Good typing skills are essential for obtaining a job as a flight crew scheduler.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}