{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e2a44719-99fd-5092-a2a9-238ce0047a30","task_key":"default--validation--43358","task_revision_id":"1","upstream_id":"43358","short_description":"HellaSwag validation 43358","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to beat a taller and bigger opponent in a street fight [title] Don't fight unless you have to. [step] When faced with a bigger opponent, you're going to be at a disadvantage from the start. Unless there's truly no other option, try to defuse or avoid physical violence at all costs.\",\"endings\":\"[\\\"[title] Don't be vulnerable and don't run. [step] If a big opponent is in front of you, chasing after them will only leave them overwhelmed and incapacitated.\\\", \\\"[substeps] Don't throw a punch unless the taller matches a larger opponent. Unless there's not much you can do, or you're already getting dirty and tiring of fighting, go ahead and punch the taller opponent.\\\", \\\"Do your best to make peace or simply walk away, if you can. There is no shame in running from a fight, especially if you don't know how it will turn out.\\\", 'The more violent your opponent, the less likely it is for them to carry out an offensive attack. [substeps] Fighting your opponent right now will probably only worsen their momentum and ultimately give you a bigger advantage.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}