{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9a239916-f7cf-53d2-9d46-73d3052b342f","task_key":"default--validation--33897","task_revision_id":"1","upstream_id":"33897","short_description":"HellaSwag validation 33897","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to spot fake all star converse [title] Check the packaging. [step] One easy way to spot a counterfeit is when the all stars come in a box that is obviously not converse. Fresh shoes also come with tissue in the packaging.\",\"endings\":\"['If you determine that the shoes are fresh, you will not have to worry about counterfeit footwear. If the packaging is reputable, the buying process will not have much effort on your part.', 'Sometimes they\\\\'ll even have paper stuffing in the shoes. If you don\\\\'t see any of the \\\" typical \\\" signs of fresh shoes, they\\\\'re probably worth suspicion.', '[title] Examine the insole of the shoe. [step] If your shoe contains a fake insole, it likely came in a box.', \\\"Look for the logo of either of these manufacturers. You don't want your shoes to be a fake made from an authentic brand.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}