{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"be8da7e3-8431-519b-bdf9-808ae6421ea9","task_key":"default--validation--25271","task_revision_id":"1","upstream_id":"25271","short_description":"HellaSwag validation 25271","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to clean yourself in the bath [title] Clean your bathtub. [step] In order to have clean bathwater, you need to scrub your tub. All you have to do is mix 3 cups of hot water with 1 cup of laundry detergent in a spray bottle.\",\"endings\":\"['Spray your tub as many times as necessary, keeping it shut as you wash your face and body. [title] Soak for 10 minutes in your tub for ten minutes.', 'Spray the solution all over your tub and scrub with a long handled bathtub cleaning tool which you can buy almost anywhere cleaning supplies are sold. Here are some other items you might want to help you clean.', 'This mixture is very cleaning, as it reduces the likelihood of germs and bacteria floating around in the water. [substeps] Fill a sink or basin with hot water.', 'Dry it with a clean, soft towel. Pour some of the water into the shower and allow it to warm up.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}