{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"01ee37ca-bf09-56d6-a408-ebe5f8e18911","task_key":"default--validation--30358","task_revision_id":"1","upstream_id":"30358","short_description":"HellaSwag validation 30358","config":"default","split":"validation","body":"{\"activity_label\":\"Beach soccer\",\"ctx\":\"Several shots are shown of people wandering on a beach that lead into clips of a soccer match and people celebrating. more people\",\"endings\":\"['are seen trying to play the game on the beach as well as running into others while the move forwards and backward and stop to speak to one another.', 'are seen dancing around as well as playing soccer and gesturing to the camera.', 'are shown doing various sports on a beach while taking photos and celebrating with others.', 'are seen playing soccer while hiding in a tent and later on being interviewed.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}