{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"924abbaa-7e80-5339-a663-9eab18b8d261","task_key":"default--validation--41207","task_revision_id":"1","upstream_id":"41207","short_description":"HellaSwag validation 41207","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to throw a kitten shower [title] Work with a local shelter or humane society. [step] If you're not already affiliated with one, contact a local animal shelter or humane society. Let them know that you're interested in throwing a kitten shower to benefit their organization.\",\"endings\":\"['They may be able to offer you helpful information regarding the upcoming kitten shower procedures. [substeps] In the united states, the shelter association (aafcc) lists shelters that accept cats.', \\\"Let them know that you're hopeful that the kitten will then cheer you up. [title] Set up a puppy shower or play outside.\\\", '[substeps] While shelters and humane societies often plan their own kitten showers, they make great projects for girl and boy scout troops, church or civic organizations, or a group of animal loving friends. A shower is typically held at the shelter or humane society it benefits.', \\\"In some places, you can ask them to send you a promotional flyer, and in others, they may even take that flyer, print it out for you, and send it to their group of shelters. [substeps] Check your state's humane society website to find shelters near you.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}