{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"82eef5c2-6cb5-59c4-9142-2530261da748","task_key":"default--validation--36335","task_revision_id":"1","upstream_id":"36335","short_description":"HellaSwag validation 36335","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to clean road salt off leather shoes [title] Use vinegar and water. [step] A great diy product for removing salt stains from leather shoes is a solution of water and vinegar. [substeps] Simply mix two parts water with one part vinegar in a small jar.\",\"endings\":\"[\\\"If you don't have vinegar readily available, use an old toothbrush to scrub a small amount of vinegar directly into the stain, making sure to splash it as often as possible. Baking soda is a very similar stain remover to vinegar but it does not wipe away the stains.\\\", 'Spray onto dry, cracked, or slightly scratched areas. Leave the shoes to soak in the mixture for several hours, then rinse and repeat when necessary.', 'Put a wooden spoon in the jar while the shoes are still in the solution. Let the shoes sit in the solution for a few minutes before removing the shoes.', 'Dip a clean, soft rag into the vinegar solution and use it to gently wipe away any salt from the surface of the shoes. Remove the vinegar solution with a water-soaked cloth, then dry with a clean towel.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}