{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"10a200f7-1fca-524b-a6cd-86b9cd590cb3","task_key":"default--validation--41642","task_revision_id":"1","upstream_id":"41642","short_description":"HellaSwag validation 41642","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to act like you saw a ghost [title] Start off by describing a dream. [step] This dream should involve a ghost; for example, you might say, \\\" i had a dream of a boy dying because he was in a car and the gas ran out in the middle of the train crossing. Then the boy was crushed by the train! [title] Elaborate on the story.\",\"endings\":\"[\\\"[step] Your outline outlines the main details of your dream in a simple, unexpected way. If it's your first time telling a story, your outline might include a jumbled up scene or a whole scene.\\\", '[step] Say that in a next dream he was talking to you and he was mad/sad or happy and warning you of something. [title] Reveal that you feel someone is watching you.', '[step] Start explaining why you know where the ghost is in your dream and why you know how to tell the ghost to go away. Remember, the following are fairy tale stories : [substeps] Dreams involving the swamp, graveyards, etc.', '[step] Let your imagination run wild and describe something that occurred in your dream. If you are in a dream where you tell a ghost story about someone scary, try to build your story around it.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}