{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"117fe2b7-c689-5f54-8b38-7d988600b7a5","task_key":"default--validation--18188","task_revision_id":"1","upstream_id":"18188","short_description":"HellaSwag validation 18188","config":"default","split":"validation","body":"{\"activity_label\":\"Canoeing\",\"ctx\":\"Two people prepare to go kayaking and are pushed into the lake. two people\",\"endings\":\"['are kayaking together and they all go down the river laughing while paddling themselves through.', 'then paddle around, then come back and celebrate.', 'swim to the bank of the water and get out of the kayak.', 'rock they row and swim in the water and the kayaks crash into a large rock.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}