{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"41863de5-363a-50e5-a9fa-85653af44f70","task_key":"default--validation--30111","task_revision_id":"1","upstream_id":"30111","short_description":"HellaSwag validation 30111","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to urinate without touching the toilet [title] Buy a funnel. [step] Women have long been searching for ways to avoid touching the toilet. It hardly seems fair that men can stand when you have to sit, right? Fortunately, the marketplace has responded to women's needs.\",\"endings\":\"['Just take a funnel and pour all the water in the bottle that came along with your prescription. Using the funnel will squeeze the water out and ensure it stays as liquid as possible.', \\\"There are now funnel-like devices for sale that allow women the ease of standing while urinating. When you're shopping, look for what is known as a female urinary device.\\\", '[substeps] To get your money, or most likely your cheapest fellow posers, consult a bathroom photography program. Paint or pastels some open in background color.', \\\"These women are not taken seriously and girls who find themselves standing around this potty seat several times a day can be left out of urine, which is why you should let other women go public. It's one of the benefits of this car, if you have more than one.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}