{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"964e7c42-18be-5cc3-a674-d286fa8e4a58","task_key":"default--validation--1767","task_revision_id":"1","upstream_id":"1767","short_description":"HellaSwag validation 1767","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to survive as an intellectually gifted person [title] Avoid making your intelligence the core of your identity. [step] It sounds impossible, but work hard to resist thinking of yourself as \\\" the smart one \\\" in your friend group. You will not always be the most intelligent person in the room, and the farther you get from school, the less book smarts will matter in your day-to-day life.\",\"endings\":\"['You should actually work to put an end to them, instead of comparing yourself to them. [title] Remember that above all else, acceptance is the key to surviving.', '[substeps] Study hard and do your best in everything you do, even if your grades drop. You will get good at reading and math.', '[substeps] There is more to you than your brains. Chances are those other things are what people like best about you.', \\\"Thus, work hard to be smart and overthink, and you'll end up attracting too much attention and spiraling into poor academic success. [substeps] Remember, it's only for appearances, not knowledge.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}