{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"47077f33-6ca4-5ca3-a140-8979c47b2d97","task_key":"default--validation--26982","task_revision_id":"1","upstream_id":"26982","short_description":"HellaSwag validation 26982","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to wash leather [title] Hand wash leather for basic maintenance and deep cleaning. [step] Hand cleaning is great for spot treating scuff marks and dirt, but it's also an effective way of deep cleaning leather. Although if your item is expensive or made of hard leather, a professional cleaner might be the safest cleaning method.\",\"endings\":\"['Hold the item in place by gently pulling down on the leather to put in a pressure washer. [substeps] You may also want to give the item a rinse or rubbing to remove dirt and grime from the surface.', '[title] Make a soapy water solution with castile soap. [step] Pour a small amount of castile soap into a bowl of distilled water.', \\\"If you can never and don't want to spend more money on a professional cleaner, contact your local grout company and ask them about the pros and cons of using this method. [title] Vacuum your leather to help remove stubborn debris.\\\", '[substeps] Before you hand wash leather, skip the wash cycle and instead do a deep clean with mild soap and water. Good deep cleaning cleansers include dawn soap, ammonia, and chlorinated water.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}