{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ab565177-5f9a-57b0-8d41-ed147b615ff1","task_key":"default--validation--29108","task_revision_id":"1","upstream_id":"29108","short_description":"HellaSwag validation 29108","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to remove kool aid from skin [title] Make toothpaste your first choice. [step] Toothpaste is generally considered the best method for removing kool aid. If you have toothpaste on hand, use that before trying other means.\",\"endings\":\"['[title] Be careful when cooking with other items. [step] Kool aid can dissolve and clog your mouth.', \\\"Stain removal or whitening toothpastes are probably your best option as they're designed to remove set in stains on teeth and will act similarly on your hands. Look for toothpaste that contains blue covarine, as this has the most immediate effect.\\\", 'You can use the same type of toothpaste used to make your toothpaste : [substeps] Mayonnaise. For instance, you could use this over a cup of some of your favorite toothpaste.', 'You can also prepare the toothpaste with either honey or yogurt; both-based stinkings do. [title] Mix the toothpaste, honey, and yogurt.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}