{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d69bd07b-980e-5cef-8768-67a38072abba","task_key":"default--validation--9843","task_revision_id":"1","upstream_id":"9843","short_description":"HellaSwag validation 9843","config":"default","split":"validation","body":"{\"activity_label\":\"Layup drill in basketball\",\"ctx\":\"The man then grabs a second ball and continues to demonstrate advanced mikan drills with both balls. when the man\",\"endings\":\"['is finished, he runs back to congratulate his balls and congratulate the others.', \\\"is done with the drills he's talking again and the outro appears with his contact information and a subscribe button while a small video shows some of his other skills.\\\", 'is done with his count, he turns the pinata and begins to pour some water.', \\\"'s moves are done, he drops two balls on the ground before piling them.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}