{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"71d4e32d-e3e8-5d20-b7ab-b4d9a0e69478","task_key":"default--validation--37057","task_revision_id":"1","upstream_id":"37057","short_description":"HellaSwag validation 37057","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to act like a hipster [title] Avoid the mall. [step] Hipsters enjoy discovering new styles and brands. They also enjoy thrift stores.\",\"endings\":\"['The mall is a great place to meet new people and expose yourself to new things. [substeps] Try to avoid the mall area as much as possible.', 'A cluttered mall is not a match for a hipster. [substeps] Go for dresses that hit mid-thigh or higher, as this shows off your figure.', \\\"Malls are seen as too mainstream and associated with authority. You don't want to buy mass produced products.\\\", \\\"If you want to get your name out there, then do so in different places. For example, you could head to a new york fashion shoestring in everyone's area, or buy your entire wardrobe from a thrift store.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}