{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f61dfdce-2c3c-5e10-aa0f-7e49ce5caa86","task_key":"default--validation--49717","task_revision_id":"1","upstream_id":"49717","short_description":"HellaSwag validation 49717","config":"default","split":"validation","body":"{\"activity_label\":\"Playing violin\",\"ctx\":\"A girl dressed in a white blouse and black skirt is standing on stage in front of an orchestra with her violin. The orchestra begins to play and close ups of the clarinets are shown. finally\",\"endings\":\"[', the girl begins to play and she is walking back and forth across the stage playing in front of hundreds of people.', ', the young singer begins to talk.', 'the clarinet is handed the person grabs the bow and begins to measure the neck of the instrument as the orchestra play.', 'the girl in black shirt dances along with the music and the temperature drops rapidly while many watch on the sides, the little girl begins to laugh and the conductor begins to kiss her.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}