{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6215f459-74ec-55c4-8377-40c9a1b58c88","task_key":"default--validation--50156","task_revision_id":"1","upstream_id":"50156","short_description":"HellaSwag validation 50156","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to clean merrell shoes [title] Brush off small amounts of dirt regularly. [step] As merrell shoes are often used for outdoor activities, they're prone to building up dirt and other debris. Make a habit of cleaning merrell shoes after each use.\",\"endings\":\"[\\\"After wearing your shoes, use a soft brush to gently brush off any dirt or debris. [substeps] If you're vigilant about brushing off dirt regularly, it's less likely to build up on your shoes.\\\", 'Use a soft bristle brush to wet the surface of the shoe. Use a smooth cloth to rub away the dirt and many large clumps of mud.', \\\"Every time you boot up the boat, take several moments to gently brush dirt off the shoes. Once you've finished this scrub, let the shoes air dry while you get dressed.\\\", 'This will help prevent dirt from getting on the soft, rubber surface of the shoes and causing blisters. [title] Run a hard-shoe brush over shoe soles twice a week for a few minutes to get rid of built-up dirt.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}