{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6713e066-6e2d-510b-95a5-a95f18cb69cf","task_key":"default--validation--1685","task_revision_id":"1","upstream_id":"1685","short_description":"HellaSwag validation 1685","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to deodorize ugg boots [title] Fill a small bowl with equal parts of baking soda and cornstarch. [step] 2 teaspoons (10 grams) of each ingredient will be plenty. Both baking soda and cornstarch have odor-absorbing properties.\",\"endings\":\"[\\\"[substeps] The recipe calls for equal parts baking soda and cornstarch, but it should take a bit of experimentation to find the right combination. If you can't find corn starch, xylitol and abc flour are just a few try adding 2/3 of it (even if you think it's too dry).\\\", \\\"[substeps] If you can't find cornstarch, use cornflour instead. Don't use cornmeal, it is not the same thing.\\\", 'This is why it is important to test the mixture before using it in baking dishes. [title] Mix all of the ingredients on a table.', \\\"You can also use a ½ cup (120 milliliters) bucket to waste the baking soda. [substeps] If you don't have anything to use, try throwing in 1 teaspoon (4.93 milliliters) of baking soda.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}