{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"5e5270ae-c0a8-5e67-95a2-f5636c93eee2","task_key":"default--validation--2627","task_revision_id":"2","upstream_id":"2627","short_description":"HellaSwag validation 2627","config":"default","split":"validation","body":"{\"activity_label\":\"Disc dog\",\"ctx\":\"Jesse is running around catching the frisbee the lady throws. We see five still shots of jesse jumping in the air. jesse\",\"endings\":\"['continues to run around with his frisbee.', 'is then shown jumping again and we see the men giving each other high fives.', 'kicks after the dog while the lady helps.', 'immediately shits in pain while having a hard time catching the frisbee.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}