{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1fc90f3f-2dae-51cd-879a-6784b4ae4b25","task_key":"default--validation--15881","task_revision_id":"1","upstream_id":"15881","short_description":"HellaSwag validation 15881","config":"default","split":"validation","body":"{\"activity_label\":\"Waterskiing\",\"ctx\":\"A group of people are seen sitting on a dock while another group rides away in a boat. The people talk back and fourth while one man sits on a pair of skis. the boat\",\"endings\":\"['continues on being shown.', 'moves through the water with people following behind.', 'takes off and the man rides in the back.', 'speeds along the water and the people follow behind the other on their snowboards.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}