{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"504041d5-b8e9-532a-9597-e538d4bd2d76","task_key":"default--validation--33563--variant-c594ece8886b","task_revision_id":"1","upstream_id":"33563--variant-c594ece8886b","short_description":"HellaSwag validation 33563","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to set a reading goal that will help accomplish your other goals [title] Decide how much to read. [step] The amount of reading you'll need to do to help you reach your goal will vary based on what your goal is. To begin with, try to develop a general sense of how much reading you'll need to do.\",\"endings\":\"[\\\"This will help create a schedule for the reading intervals. [substeps] For example, if you have a goal to read 50 pages per day, then let's say you have 25 and 30 pages of reading each day.\\\", 'This will drive the rest of your planning. [substeps] For example, if your goal is to recognize edible plants in your area, one or two good books on the subject will probably be adequate.', \\\"It's better to determine the exact amount of reading you'll need to do, rather than let feelings of interest get the best of you. [substeps] If you're reading for two columns, start by reading a scale from 135 to 140.\\\", \\\"If you have reading goals, be sure to think about these in a given day. [substeps] For example, if reading is an enjoyable activity, add in what you need to study and how much you'll need to do.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}