{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"72eec70c-6664-54bc-a42e-7c93ebe9bed6","task_key":"default--validation--38350","task_revision_id":"1","upstream_id":"38350","short_description":"HellaSwag validation 38350","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to unclog a slow running bathroom sink drain [title] Gather your materials. [step] Instead of relying on drain cleaner products, which are often corrosive and can cause allergic reactions and respiratory problems, you can use household items you likely already have. You will need : [substeps] Rags baking soda vinegar lemon boiling water [title] Measure out your ingredients.\",\"endings\":\"['[step] A predraining drain about a quarter-inch (0.64-centimeter) in diameter. Washing sink drain cleaner products by hand is a typical method, but you can buy them at most home improvement stores.', '[step] Your base needs is flour, baking soda, white vinegar, and water. You can use a measuring cup to measure them out, or even a tea kettle.', '[step] Take ¼ cup of baking soda , 1 cup of white vinegar, and 1 large pot of water to boil. Have a rag or sink stopper handy.', '[step] While not all small kitchen sinks are clogged, hot water can help clear the clogged holes. Using a measuring cup to measure out your ingredients is particularly important, since boiling water will also introduce debris such as body matter, clay, and even animal waste.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}