{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2accc6cd-7ad1-5de5-b2d8-a942c32cee8f","task_key":"default--validation--13160","task_revision_id":"1","upstream_id":"13160","short_description":"HellaSwag validation 13160","config":"default","split":"validation","body":"{\"activity_label\":\"Running a marathon\",\"ctx\":\"A large group of people are seen standing behind one another with others watching on the sides and leads into the group running down a road. the large group of kids\",\"endings\":\"['then stand still and each takes turns running down a different road several times as the people watching continue to laugh and then come back down the road in the end.', 'continue running down the road after one another while people still watch on the sides.', 'then play hockey together while one another run all around and end by off scoring a goal and the camera panning around to people walking around and looking at the score.', 'continues when many close up do short stunts on hole slides in the street while the camera captures them from several angles.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}