{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7aeee386-af3d-5fbc-9384-c4f689f9fe83","task_key":"default--validation--8351","task_revision_id":"1","upstream_id":"8351","short_description":"HellaSwag validation 8351","config":"default","split":"validation","body":"{\"activity_label\":\"Surfing\",\"ctx\":\"A few surfers are out in the ocean surfing. A surfer in a wetsuit rides his surfboard on the waves. the waves\",\"endings\":\"['part as he passes an old buck.', 'push the surfer into the side of the water and away from him.', 'knock him off his surfboard.', \\\"are so strong and violent and he can't control himself as he falls for a while, doing tricks and pulling other surfers like a buoy.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}