{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"96124d5a-8b76-5f7d-a615-c8800d205b16","task_key":"default--validation--5687","task_revision_id":"1","upstream_id":"5687","short_description":"HellaSwag validation 5687","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to have fashionable budget friendly gothic clothes [title] Do your research on what \\\" goth \\\" really means. [step] There are many different looks within this subculture (cyber goth, romantic goth, renaissance goth, etc), and to create an extensive wardrobe you will need to figure out exactly what look you are aiming for. [title] Once you have decided on a style that suits you, go out and shop for the basics.\",\"endings\":\"['[step] Go to etsy, mosties, and scene magazines ;. Nothing dresses up gothic! [title] Head to the stairs.', \\\"[step] Get the basics that make up the gothic stuff. You'll need the following things : [substeps] A uniform.\\\", '[step] You can just get the basics to go, but this can be quite pricey. You can get cash for a wardrobe change, or custom clothes.', '[step] Basics include things like plain black pants, black t-shirts, a black jacket, and black gloves for those chilly days. [title] Next, you will need to accessorize.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}