{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"140a23e8-f861-53db-ac54-fab0ad86bc51","task_key":"default--validation--13950","task_revision_id":"1","upstream_id":"13950","short_description":"HellaSwag validation 13950","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to win nanowrimo [title] Decide if you are a planner or pantser. [step] In the nanowrimo community, there are two types of writers. The planners are those who plan their novel ahead of time.\",\"endings\":\"['The pantser authorizes everyone to write a manuscript in advance. While many writers prefer to write a draft of their stories before preparing a draft of their own story, individual writing projects can quickly take a toll on these creative pursuits.', 'Not all authors decide early on that they want to write a novel after 10 days, but some writers may opt to write 30 days after the last day they plan to write. [substeps] The planners are the people who set the project down in advance.', 'The planners, on the other hand, do not undertake projects at a much later date or in the future and perform content except for experiments. If you are not a planner, then an ultimate planner, which lists designs and content, is more like an online accounting expert.', 'The pantsers are those who wing it as they go along. Think about how prepared you want to be before you start writing your novel.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}