{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"92d296ae-848e-5717-8f27-ffcd8ff50ca2","task_key":"default--validation--33530","task_revision_id":"1","upstream_id":"33530","short_description":"HellaSwag validation 33530","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to fight off a dog [title] Stay on your feet and shout for help. [step] If you let the dog jump on you or bring you to the ground, you'll be in a much weaker position and more likely to be seriously hurt. If you're in a city or residential neighborhood, call out for the dog's owner.\",\"endings\":\"[\\\"If you're in the woods or a rural environment, you'll need to call out for any help you can get. [title] Place an object between yourself and the attacking dog.\\\", 'If you are near a phone or an emergency room, call for help from the police. [title] Pull as far away from the dog as possible.', '[title] Call out for help if a dog nips at your wrist. [step] If you have the option of fighting your dog, acting calmly and without aggression will help.', 'This way, the attacker might not be able to strike the dog if it is hurt, and the dog will be off your back. [substeps] If possible, run behind a building to minimize the possibility of injury.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}