{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0c3a01a5-f428-510d-97f0-ec659e61d862","task_key":"default--validation--33018","task_revision_id":"1","upstream_id":"33018","short_description":"HellaSwag validation 33018","config":"default","split":"validation","body":"{\"activity_label\":\"Ballet\",\"ctx\":\"A woman dressed in a blue flowing dress with a blue bodysuit underneath is standing up, and she begins to do a ballet dance routine in the middle of the large dancing floor while the people in the reception area are watching her every move, taking picture and videoing her. when the woman\",\"endings\":\"['is done dancing she stands there with her arms crossed on her chest and the crowds applauds, and gives her a standing ovation while she takes a few bows.', \\\"is done dancing, she lifts her face mask and starts to laugh off into the distance and when she removes the mask, there's a man behind her who is also laughing off camera in a hotel room, then there's the body suits laughing off camera in a hotel room.\\\", 'is finished with the dance, a man shows up to help and they all both begin to leave the floor, then there is a pause in the choreography.', 'is done while all the people are still watching, a newscaster wearing a blue dress comes onto the screen and begins talking and showing a newspaper with a logo.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}