{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3ddd652c-4091-519b-9463-f428fb5afb8f","task_key":"default--validation--37839","task_revision_id":"1","upstream_id":"37839","short_description":"HellaSwag validation 37839","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to name a main character [title] Choose time-period appropriate names. [step] Think about what year the story is set in, and the age of your character. Pick a name that is suitable for the era of the story.\",\"endings\":\"['For example, having a story set in 1998 might be humorous for the time period, but in modern times it might be something more. For example, the great dane from 2007 may have his dream baby stuffed animal in his closet.', 'Names that are currently popular may have been rare or not used at all in the past. [substeps] Yearbooks and old newspapers are a great place to find era-specific names.', '[substeps] For example, there are 252 characters in our story, and none of these characters would be fictional. For example, what type of parents would your character be if they were two ordinary people who lived hundreds of years ago? [title] Decide where to name your characters.', 'If your character is a werewolf or vampire, your term of name could be \\\" werewolf. \\\" if your character is vampire, pick a night (which should be fairly dark) time period with ample daytime hours, preferably not at night.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}