{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cfa8902c-7bc7-5f1e-b203-8d8b595dbf3d","task_key":"default--validation--17088","task_revision_id":"1","upstream_id":"17088","short_description":"HellaSwag validation 17088","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to be thorough in your oral hygiene routine [title] Clean your teeth at least twice a day. [step] The best oral hygiene habit to get into is to brush your teeth at least twice everyday-once when you wake up in the morning, and again before you go to bed at night. If you do absolutely nothing else for your teeth, at least do this.\",\"endings\":\"['For your other hygiene duties (like keeping your teeth clean, eating healthy, taking a shower, etc. ), do the following : [substeps] Gently suck on a paper clip or pencil.', '[title] Wait at least one hour after eating to brush your teeth. [step] When you eat and drink during a meal, the enamel on your teeth can soften temporarily.', \\\"[substeps] Check the inside of your mouth to make sure you don't have anything in your mouth that is also stinky or bad smelling. If you do have some, don't put that into your mouth.\\\", '[title] If you are going to be exercising, be sure to stretch, too. [step] At regular intervals throughout the day, you should stretch up and down to stretch out every muscle involved in exercising.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}