{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0f687d35-59ca-5ea3-9436-650ee69df829","task_key":"default--validation--49718","task_revision_id":"1","upstream_id":"49718","short_description":"HellaSwag validation 49718","config":"default","split":"validation","body":"{\"activity_label\":\"Playing violin\",\"ctx\":\"Finally, the girl begins to play and she is walking back and forth across the stage playing in front of hundreds of people. the violinist\",\"endings\":\"['then begins to hit the keys and used her hands and hips a little to basically score with her hands and the keyboard.', 'makes two pirouettes in place before finally hitting the baton and being led to another spot.', 'fourteens to the end and the show ends with advertisements shown on the screen.', 'completes her piece and the host who is dressed in a red gown comes out with a mic and begins talking before bowing to the crowd.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}