{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f0a378f7-92b5-5581-a96d-7ef74bd9155e","task_key":"default--validation--1495","task_revision_id":"1","upstream_id":"1495","short_description":"HellaSwag validation 1495","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to analyze a historical document [title] Determine what type of document it is. [step] It's important to know what you're looking at, since that can help you figure out the document's context. You could be looking at a newspaper article, a music score, a piece of poetry, a letter, or something else.\",\"endings\":\"['Don\\\\'t confuse \\\" why is a footnote \\\" with \\\" write a footnote. \\\" it may seem like the sole purpose of a footnote is to explain something, and writing a footnote could possibly contain much more information than that.', 'Either consider the format of the document or its subject matter. Here are some key things to consider when deciding what format you should use when determining the format of the document : [substeps] If it is a word, it likely indicates a period.', \\\"Knowing what you're looking at helps you develop an approach for handling the information the document contains. [substeps] To determine what kind of document it is, study it closely and consider where it came from.\\\", \\\"Don't keep in mind what just happens to be a historical event, so all you can do is get your ideas in order. [substeps] Many historical publishers use copyright law to publish scholarly articles.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}