{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"901e7be4-cac2-5ccf-a19f-833b6ac72270","task_key":"default--validation--38628","task_revision_id":"1","upstream_id":"38628","short_description":"HellaSwag validation 38628","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to write apocalyptic stories [title] Decide what kind of apocalyptic story it is : [step] Before the world ended, while the world ended, or after the world ended. [title] Decide why the world ended and how : [step] Plague, nuclear war, aliens contact, etc. [title] Decide how your character (s) survived the apocalypse (e.g. [step] Immune).\",\"endings\":\"['[title] Choose what magic you want to use. [step] Does your character (s) have human characteristics or is he more (non-human)? What sort of to use? [title] Decide what genres (gothic, fantasy, goth, sci-fi, etc.', \\\"If you don't, you'll leave major holes in your story. If you're writing a story placed before or during the apocalypse, you can use a character's survival method to your advantage by making it the story line.\\\", \\\"If you enjoy your fictional world a bit more intensely, and plan to repeat the story where it ended, then you're good to go. If you don't and can no longer enjoy it, then you're heading for serious trouble with apocalypse.\\\", \\\"[title] Send me a description of the ending and how it affected the reader's perception and thinking at the time. [title] Write a foreshadowing of what happens next.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}