{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b7da1e23-13d5-59a6-a036-e4d65d89ce66","task_key":"default--validation--39832","task_revision_id":"1","upstream_id":"39832","short_description":"HellaSwag validation 39832","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wear a camel coat [title] Choose a trench coat style for a formal look. [step] Camel trench coats are one of the most popular styles, because they can make any outfit look immediately classier. Fortunately, you can find ones in a range of materials from light cotton to heavy wool, depending on what you need for the fall weather where you live.\",\"endings\":\"['[substeps] Camel jackets are typically black-gray, velvet, or beige. They have multiple sleeves in a row, and extend from the upper arms to the lower legs.', \\\"You'll just need to choose one or both for all seasons. [substeps] Coat coats in light cotton are most commonly worn in the summertime, while coats in heavy wool are most often worn during the winter or during hot, sunny months.\\\", '[substeps] The main reason camel trench coats are common was that they are far more casual than the traditionally worn more formal coats. Therefore, they take a great deal of the emphasis out of the outfit.', \\\"[substeps] Choose a coat with a belt for the more traditional trench coat style, or find a design that doesn't use a belt, giving the overall effect a less busy finish. [title] Buy a peacoat for versatility.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}