{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"03315c72-73f0-5ae7-951a-eef44a958197","task_key":"default--validation--46504","task_revision_id":"1","upstream_id":"46504","short_description":"HellaSwag validation 46504","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to write a simple screenplay (for kids ) [title] Write or type the name of the play on a piece of paper. [step] When thinking of a name, make sure that you do not plagiarize the name of a book or popular poem. Think of what you want the play to be about before you name it; this will help guide you.\",\"endings\":\"['[substeps] You can also use different names for different characters in the play, just in case your alter does not like the name \\\" barbie. \\\" do not pronounce \\\" barbie \\\" and \\\" jane \\\" altogether because this would confuse your reader.', '[title] Write down a list of actors and their parts. [step] Do not worry about thinking of a lot of characters, and focus on creating good quality characters.', '[substeps] For a simple, name the play on a piece of paper. Write the names of the characters and authors on the top of the paper.', 'Some things to think about are : [substeps] The books or poems you want the play to look like. The genre (books like \\\" seventeen \\\", \\\" alice who ate a prawn cricket horse \\\").']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}