{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6a19eb71-de77-5178-b9f7-82100c1ac4cf","task_key":"default--validation--22514","task_revision_id":"1","upstream_id":"22514","short_description":"HellaSwag validation 22514","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to not get hurt in a fight [title] Recognize aggressive behavior. [step] If someone is trying to provoke you, don't challenge them. [title] Be ready.\",\"endings\":\"['[step] If a fight gets out of control, you will be more likely to trip on your way out. This is because physical impact and potentially dangerous contact can trip you up, especially if you are armed with a handgun.', '[step] When you see the force coming on you, stay calm. If you are extremely defensive, it may be too late.', \\\"[step] Don't let them take you by surprise. If you're in a situation where you think someone might attack you, stay vigilant.\\\", \\\"[step] If it's your first fight with a male bully, recognize that he'll punch and raise his hands before he's taken a power punch. If he goes into you without a weapon, there's a possibility he'll hurt you again.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}