{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e8e82599-5333-5c38-9294-8e04bd7c02b3","task_key":"default--validation--33305","task_revision_id":"1","upstream_id":"33305","short_description":"HellaSwag validation 33305","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to be a wholesome american girl [title] Wholesome girls are clean. [step] Always shower and wash your face. Keep your skin and hair greaseless.\",\"endings\":\"['Brush your teeth at least twice a day with whitening toothpaste for clean pearly whites. Clean hair, skin, teeth and nails are a must.', 'Wash your legs and then, touch them in areas where oil often gets applied. You should also wash your ears, nose, lips, and eyelids, with lotion and fresh washed feet.', \\\"Drink enough water if you prefer to look clean. One way to look wholesome is to smoke, smoke straight up, and go for a snack whenever it's hot out.\\\", \\\"Plus, it doesn't really matters if you wash your hair, because you can get pimples. Write these tips down in a cute notebook, a diary, and perhaps even your bedroom wall.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}