{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d3cf7e51-76ae-5c95-bb72-f05667d02cae","task_key":"default--validation--6953","task_revision_id":"1","upstream_id":"6953","short_description":"HellaSwag validation 6953","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to make posters for a school election [title] Figure out the number of posters you are allowed to hang. [step] Try asking the person who is in charge of the school elections or the principal. Choose a time when they are not busy and, if you easily forget things, bring a notebook and pen to take notes.\",\"endings\":\"['[substeps] If you have a locker, try to choose a day when the people in charge of the school elections are not busy, such as recess and the winter holidays. During the school election, ensure that posters should not be lost.', '[title] Figure out how many posters you must have. [step] The poster must be at least 8 inches by 8 inches (20.', \\\"A quick internet search is always good to get a good idea of the number of votes you could raise. [title] Find these posters and use these to put signs on schools' doors.\\\", 'Ask for other specific details, such as where/not you are allowed to hang the posters. Also, ask what they would like you use to hang the posters.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}