{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cd9fbe87-42da-539f-933b-91080143704b","task_key":"default--validation--46898","task_revision_id":"1","upstream_id":"46898","short_description":"HellaSwag validation 46898","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to write a book summary [title] Annotate the text as you read. [step] Keep a notebook next to you while you read so you can jot down your thoughts. Taking notes as you read will allow you to record things correctly.\",\"endings\":\"['[substeps] Do not turn away or glance down at the text for long periods of time. Look out for typical pauses or formatting as you read.', \\\"If you're writing about a project or a situation, label the text in the notebook so you can easily identify it later. [substeps] Avoid using bullet points to record the text, since these can take up too much time.\\\", 'This will create less work for you than if you try to go back and check on details later. [substeps] It might be a good idea to keep several different pieces of paper next to you for notes.', 'Even if you are only reading a small portion of the text, it is still important to have an important section of the text highlighted. [substeps] Annotate up to five chapters in a page.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}