{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9bd9fad1-29eb-5ec9-a65c-3c5b0fa7a855","task_key":"default--validation--18794","task_revision_id":"1","upstream_id":"18794","short_description":"HellaSwag validation 18794","config":"default","split":"validation","body":"{\"activity_label\":\"Disc dog\",\"ctx\":\"A woman in a pink jacket and her dog are outdoors and doing tricks wish discs as she throws them the dog catches, as well as the dog jumping over her, rolling over, dancing. a still shot\",\"endings\":\"['is being shown of the dog when a man comes along and the woman walks away walking with friends in the background.', 'appears of the woman and her dog as the 1st place winners at the championship.', 'video of her throwing disc gives the information about the frisbee to the dog.', 'video scores and one famous cliesman speaks to the camera.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}