{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"948519b5-ead7-5df5-8f06-b6d04aa74ed8","task_key":"default--validation--24228--variant-ac80fd6eeef4","task_revision_id":"1","upstream_id":"24228--variant-ac80fd6eeef4","short_description":"HellaSwag validation 24228","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to avoid hurting your gums [title] Have a professional bleach your teeth. [step] Bleaching your teeth can help whiten your teeth, removing stains from food and drinks. You may have noticed bleaching kits at the drug store.\",\"endings\":\"[\\\"Or, simply see your dentist if you are not sure if this is the best way to whiten your teeth. You should never go down to the doctor's office and have them do a bleaching of your teeth.\\\", '[substeps] If you have any exfoliants or cleansers on hand, make sure you have them before you start bleaching your teeth. [title] Use products that contain fluoride.', 'These are labeled as dental whitening strips. They may also be used to put on your shoes, and may leave you with urine-like stains that are not obvious on your shoes.', \\\"However, it's a better idea to have your dentist bleach your teeth. For one, it's usually more effective.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}