{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5bd21c91-b069-5e1c-8d6c-08d4be01d13f","task_key":"default--validation--44090--variant-615d9c47f366","task_revision_id":"1","upstream_id":"44090--variant-615d9c47f366","short_description":"HellaSwag validation 44090","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to use parallelism correctly [title] First, consider what types of items you are listing. [step] Are you listing things or ideas? Are you making a list of what a person is doing? To use parallelism correctly, you want the list to include the same parts of speech in the same form. [substeps] The part of speech is the name given to each word to describe its function.\",\"endings\":\"['For example, if someone is playing music, \\\" yuh, yuh, yuh, yuh, yuh. \\\" [title] See if you are listing things in the different forms of the word.', 'For instance, a noun is a person, place, idea, or thing. It does action or has action done to it, sometimes.', 'For example, \\\" john and craig cross are working together to make breakthroughs in superman and superman\\\\'s achievements. They both believe in superman and make it in superman.', 'Define how each individual speech followed by a comma. For example, you would write \\\" someone, george, has 30 words for speech.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}