{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"999f65ed-f6ee-5440-ab22-f4e99e7e98af","task_key":"default--validation--9653","task_revision_id":"1","upstream_id":"9653","short_description":"HellaSwag validation 9653","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to be a redneck [title] Wear practical clothes, not stylish clothes. [step] If you're willing to get deer blood and mud on whatever you're wearing, then you're dressed good enough for a redneck. Redneck life is about being prepared and capable for what comes at you, not looking any particular way.\",\"endings\":\"['[substeps] Dress good and while being redneck means not wearing other people\\\\'s clothes, it\\\\'s still about dressing to impress. Wear suit pants and dress shirts for the types of parties you\\\\'ll be the \\\" or \\\" in.', 'There are some associated styles, brands, and \\\" looks \\\" with being a redneck, but mostly that just comes from those looks being functional. [substeps] Try to wear \\\" work clothes \\\" as much as possible.', \\\"[substeps] From the 1950s, knee-high boots and a workout shirt made from recycled materials can really to waive testosterone production. If you're comfortable with your costume, wear it.\\\", 'Costumes, old sweaters and shoes are not. Wyoming is famous for tough, cut-or-kink, goth, and the punker, and you need your income and skill level to even be in the country.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}