{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"954f0585-a3ba-537d-9c07-51bcb42bc4b2","task_key":"default--validation--2119","task_revision_id":"1","upstream_id":"2119","short_description":"HellaSwag validation 2119","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to escape from being tied up [title] Position yourself to avoid being tied tightly. [step] The more loosely you're bound, the easier it will be for you to escape. Resist letting your captors tie you as tightly as possible.\",\"endings\":\"[\\\"[substeps] You will be able to remove your hands significantly tighter if you're bound tightly. [title] Look for a place to escape if you can't escape.\\\", \\\"[substeps] If you can, try to rest your hands on the front of your victim's shin. If you have any other means of concealing yourself, try keeping some of your hands locked with yours.\\\", \\\"For example, if your wrists are being tied in front of you, hold your knuckles from both hands together, and pull your hands in towards your chest. Although it looks like you're cooperating, this gesture will create a gap between your wrists.\\\", 'This prevents you from thrashing around and injuring your prisoner. [substeps] If you are forced to worry about tripping or falling, cover your torso and other loose parts of your body to avoid getting caught.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}