{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2581354b-6550-5a3d-a14b-86d18e5f325e","task_key":"default--validation--40491","task_revision_id":"1","upstream_id":"40491","short_description":"HellaSwag validation 40491","config":"default","split":"validation","body":"{\"activity_label\":\"Scuba diving\",\"ctx\":\"The focus goes back to the snorkelers as they continue to explore the area with their flashlights. the outro video\",\"endings\":\"['shows the snorkeling results on the snorkelings.', 'shows the boxes added to the video to show a fun time on the snorkeling trip.', \\\"appears and it's the same blue picture as the intro and includes some contact information on the video and the credits begin to roll with the text scrolling from bottom to top.\\\", 'leads into them walking around and surpassing some unfortunate individuals in the process.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}