{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2d1fe638-f722-598f-bf42-092629218f16","task_key":"default--validation--27716","task_revision_id":"1","upstream_id":"27716","short_description":"HellaSwag validation 27716","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to survive a deadline [title] Stop procrastinating. [step] Deadlines wouldn't be so bad if the natural human tendency wasn't to put it off until the last minute. Get started early.\",\"endings\":\"[\\\"[substeps] Mark off important deadlines on your calendar and see which ones you stick to throughout the week. This will help trick your mind into thinking you're actually getting something done.\\\", \\\"Don't let your procrastination time get too long. [substeps] You don't want to go into work already thinking about what's important.\\\", 'Work on it a little bit at a time. Most of the time, deadlines are very manageable.', 'Rather than rushing out there in the middle of the day right after taking a nap, stay in as long as you can without feeling the need to go. Commit to the deadline for at least a month.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}