{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"51542412-2d54-5a46-b6d6-6a5255353961","task_key":"default--validation--18045","task_revision_id":"1","upstream_id":"18045","short_description":"HellaSwag validation 18045","config":"default","split":"validation","body":"{\"activity_label\":\"High jump\",\"ctx\":\"A large outdoor arena is show from a view above and a display screen shows the stats for \\\"women's team standings\\\" and then the words high jump appear. a man named bryan mcbride\",\"endings\":\"['introduces a routine, he runs on the parallel bars and then runs back, jumps off of the bar and slides to the mat.', 'appears in the screen walking along the sand onto an exercise set.', 'is standing and then begins his high jump where he clears it, lands and vigorously cheers as he runs off.', 'speaks to the camera while stops to do a pose to adjust something on the saddle and then mounts the horse.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}