{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e1a66241-a3ab-50ce-aabe-d1c7c425bcb2","task_key":"default--validation--49032","task_revision_id":"1","upstream_id":"49032","short_description":"HellaSwag validation 49032","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to clean and glaze a fur coat [title] Check the coat for damage. [step] Examine the fur coat, looking for rips, tears, stains, or anything that can be further damaged during cleaning. Some damage may not be visible to the eye, so run your fingers along the edge of the lining, the collar, the armpits, the elbows and the shoulders.\",\"endings\":\"['Especially by the tip of the coat, check for solid or pitted parts. If you notice any strange stains, dash it with liquid soap or a little bit of water.', 'These are the parts that tear the most often and can be difficult to see. Check the pockets as well to make sure nothing is in there that can cause damage during cleaning.', 'There may be specific tear marks along the coat, but these are uncommon. Tear marks can indicate further damage during cleaning.', \\\"Look at eyeglass windows, floor boards, walls, trees and/or doors to make sure they aren't worn down due to wear or tear. Bowls of water and soap or washing products are also potential indications of wear or tear.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}