{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c5919199-fbd7-5274-b4dd-74c2e0169762","task_key":"default--validation--19360","task_revision_id":"1","upstream_id":"19360","short_description":"HellaSwag validation 19360","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to calm down an autistic person [title] If the person is verbal, ask them what is bothering them. [step] If it's an advertisement on a television, or a loud noise, take them away from the area (move them somewhere quiet). [substeps] During severe sensory overload, people who are ordinarily verbal may suddenly lose the ability to speak.\",\"endings\":\"[\\\"If so, they may react violently or become extremely frantic. Try to tell them you can't understand them.\\\", 'This may be due to an initial sensory overload, or because of what is going on in their head. For example, if you hear someone say \\\" oh, no...don\\\\'t, \\\" try talking a bit louder.', 'Or, their voice may become strained or hoarse. If you suspect a person is nervous or confused, tell them to take a deep breath, hold them there, and focus on their breath.', 'This is due to severe overstimulation, and will pass with relaxation time. If someone has lost the ability to speak, ask only yes/no questions that they can answer with thumbs up/thumbs down.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}