{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3117300e-701c-558e-8484-0822af5a93a7","task_key":"default--validation--23620--variant-e5e70884c34f","task_revision_id":"1","upstream_id":"23620--variant-e5e70884c34f","short_description":"HellaSwag validation 23620","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to deal with braces [title] Learn to brush your teeth carefully. [step] Brushing your teeth changes when you have braces. After getting your braces put on, learn to brush your teeth carefully.\",\"endings\":\"['Brushing your teeth for a full 60 seconds every three weeks will help prevent fluoride build up in your mouth, tooth enamel, and gum. However, brushing regularly can also put you at risk for serious health issues, especially gingivitis, peptic ulcers, and tooth disease.', 'Brush your teeth for a full 20 seconds, including the back teeth, from front to back, to get rid of any plaque and bits of debris. Brush carefully and in circular motions to get rid of sticky or foreign objects.', 'This will help you maintain the health of your teeth and teeth, as well as prevent other complications such as dental issues like gingivitis and tumors. [substeps] Brushing your teeth for about two minutes every three months can help ensure a healthy mouth and short term dental health.', 'Talk to your orthodontist about how to best brush your teeth and follow their instructions carefully. With braces, you brush each tooth carefully from the top to the bottom.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}