{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"36fba409-4bb4-5616-b7f1-2bf145397889","task_key":"default--validation--41893","task_revision_id":"1","upstream_id":"41893","short_description":"HellaSwag validation 41893","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to dye canvas shoes [title] Decide what color (or colors) to use. [step] Pastel colors show up nicely on white shoes and give nice muted tones. On the other hand, strong, bright colors also work well and will probably show up better from a distance.\",\"endings\":\"[\\\"Yellow can seem boring or unprofessional, but is generally a more powerful look. [substeps] The color wheel is used to determine color, but it isn't widely understood.\\\", 'Mix it up if you want to bleach it, but keep in mind that pastel colors tend to pop less and will take longer to react with bleach as well. Once again, it will be much harder to choose colors.', 'The choice is up to you! [substeps] Remember that the color on the dye packaging may not be exactly the same color that ends up on your shoes. With home dying you need to be prepared for a little variation and fun surprises when it comes to coloring.', '[substeps] A half-circle almost square canvas shoe works best. Or, using a pencil, trace the shape of the shoe.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}