{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4959fd1a-ae4c-5d7f-8d29-aa8675eb0230","task_key":"default--validation--15640","task_revision_id":"1","upstream_id":"15640","short_description":"HellaSwag validation 15640","config":"default","split":"validation","body":"{\"activity_label\":\"Croquet\",\"ctx\":\"This person is playing a solo game of croquet outside in a grassy area. there\",\"endings\":\"['are seven or eight small children doing the game with green balls on a chalk board under umbrellas.', \\\"'s lots of clouds in the sky and he pushes the purple ball through the ring first.\\\", 'appears to be a lot of people panning around the market, looking at the croquet game, the people on the walkway, and around the backyard, observing and interacting with the croquet game with various participants present.', 'are a few other players and one team of players is playing cricket.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}