{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"565c7400-97a7-51f0-8785-a1e9a44187a5","task_key":"default--validation--22838","task_revision_id":"1","upstream_id":"22838","short_description":"HellaSwag validation 22838","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to stop hesitating [title] Start small. [step] Commit to what you can do right now. If you know you have trouble running more than a mile, start with what you can do.\",\"endings\":\"['That is, instead of saying, \\\" i\\\\'m going to start running 4 miles tomorrow, \\\" start with \\\" i will run a mile tomorrow. Each day, i\\\\'ll try to run a little more than i did the day before.', 'Practice by running every day, and taking these stairs to work away some things. [substeps] Write down your symptoms, discuss them with a family member, or take a class.', 'You are less likely to make mistakes on the ice if you have a reason to study. [substeps] Prepare to do drills at different times.', \\\"Who knows, maybe it will start keeping you up. If you can do small amounts of exercise every day, that doesn't mean you have a problem.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}