{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0f835122-5659-515a-aa04-13f234daa3a1","task_key":"default--validation--33959","task_revision_id":"1","upstream_id":"33959","short_description":"HellaSwag validation 33959","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to look clean [title] Brush your teeth. [step] Your teeth bear the brunt of a lot of abuse and gunk over the course of your day. Between food and other factors, it's important to keep your teeth clean and white.\",\"endings\":\"['Brush your teeth after doing something with junk food and after you eat. This can make your teeth look whiter.', 'This will prevent plaque, bacteria, and oils from building up in your mouth over time. [substeps] For all people, brushing your teeth once or twice a day at least is a good way to keep your teeth clean.', \\\"[substeps] Swish a few drops of mouthwash in your mouth to clean your teeth and remove some of the gunk. It's also wise to brush your teeth after every meal to prevent them from becoming dull and gritty from trying to eat.\\\", 'Brush your teeth twice a day; once in the morning, and another time before going to sleep. Take at least two minutes to brush your teeth.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}