{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a0a43429-bb64-577d-91ac-4c1824524709","task_key":"default--validation--45662","task_revision_id":"1","upstream_id":"45662","short_description":"HellaSwag validation 45662","config":"default","split":"validation","body":"{\"activity_label\":\"Snow tubing\",\"ctx\":\"Two people from the group are dressed in winter gear, go down the slope in their tubes. the other two group members\",\"endings\":\"['follow them in their tubes as they go down the mountain slope.', 'go down the hill together, one appears to be the right ddler and the other is the espeaker.', 'get into the water in their tubes and start paddling down the hill.', 'are dressed in winter gear, the first group grabs ropes and walks together to the water and falls.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}