{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"800a74fd-8c95-5a9e-8ed4-311e97a6b012","task_key":"default--validation--13739","task_revision_id":"1","upstream_id":"13739","short_description":"HellaSwag validation 13739","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to clean a silk tie [title] Scrape off spilled food or sauce and blot the stain. [step] Use a spoon or butter knife to lift food off your tie. Then, dip a cloth napkin into water or club soda and blot at the leftover stain.\",\"endings\":\"['Continue to blot until all of the sauce and food spills are removed from your tie. [substeps] Leave your tie covered overnight or have someone help you remove the food from your tie.', \\\"If the stain is organic, you may also blot at a fruit or vegetable stain. If a star-shaped blockage like a ketchup stain is a result of your tie's clasp mechanism, take the cloth napkin and gently blot at the stain.\\\", 'As soon as you see the stain turn wet, blot at the stain again with another napkin. [substeps] This method may also work on a non-stained tie, but it is less effective.', \\\"[substeps] Paper napkins might disintegrate when you blot, but you can use them if you don't have a cloth napkin on hand. For a stain from a liquid-based sauce, such as soy sauce, simply skip right to blotting with the cloth napkin.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}