{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"5ca4ef0b-2c30-568f-a308-0720db547d56","task_key":"default--validation--1932","task_revision_id":"2","upstream_id":"1932","short_description":"HellaSwag validation 1932","config":"default","split":"validation","body":"{\"activity_label\":\"Drinking beer\",\"ctx\":\"Women serves drinks from a fountain. A man talks on front a microphone and people continue singing and showing beer. then\",\"endings\":\"[', people are again singing, drinking and dancing outdoors.', \\\", people play soccer, ripping each other's hair.\\\", ', people sit and play foosball and are shown during the show.', ', two men fight sand with an accordion while a man help him pretend.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}