{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6f4c2804-074f-5241-bae4-a823ce4e18fc","task_key":"default--validation--44137","task_revision_id":"1","upstream_id":"44137","short_description":"HellaSwag validation 44137","config":"default","split":"validation","body":"{\"activity_label\":\"Tango\",\"ctx\":\"They begin showing different steps of the dance as they talk. They move forward and back and forth, demonstrating the tango. they\",\"endings\":\"['back away and take from each others hands like this is one of them that is born to dance alone or alone and arm wresting.', 'continue moving forward as they talk.', 'kick their legs up and down, showing kicks and moves you can do.', 'then dance together for the camera.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}