{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f693cb7f-b163-562b-a272-a6e454955851","task_key":"default--validation--8184","task_revision_id":"1","upstream_id":"8184","short_description":"HellaSwag validation 8184","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make a homemade (natural) protein hair mask [title] Mix the egg yolk and yogurt. [step] In a small bowl, beat the yolk from 1 egg. Next, add 6 tablespoons of plain yogurt, and use a spoon to mix until the ingredients are fully combined.\",\"endings\":\"['[title] Add the rest of the ingredients, except for the milk and oil. [step] Once the amount of ingredients have been mixed, mix in approximately 2 egg yolks.', '[title] Combine the walnuts and coconut oil. [step] Pour the mashed walnuts and coconut oil into a separate bowl and place it in the electric mixer with the speed dial on medium for about 3 minutes.', 'Place the mixture into a pan on medium heat and cook, stirring occasionally. [substeps] When the egg yolk mixture comes to a boil, stir in 2 tablespoons of powdered sugar.', '[substeps] Egg yolk is rich in proteins and fats, so it helps strengthen and moisturize dry, brittle hair. In addition to protein, yogurt contains lactic acid, which helps remove dirt and product buildup while also moisturizing the hair.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}