{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"00155013-1dcc-5694-a4c1-cb094aed2e09","task_key":"default--validation--9059--variant-639e14f3460b","task_revision_id":"1","upstream_id":"9059--variant-639e14f3460b","short_description":"HellaSwag validation 9059","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to clean pearls [title] Mix a mild dishwashing detergent with water. [step] You can wash pearls in warm, soapy water, but it's important to use a mild detergent. Add 1 teaspoon (5 ml) of soap to 1 quart (946 ml) of warm water to dilute it, and mix well to create a soapy solution.\",\"endings\":\"['[substeps] You can use any kind of soap to clean pearls, from dish soap to dish detergent to baby shampoo. You can even prepare the mixture specially to wash pearls by hand if you want to avoid unnecessary heating.', 'Ensure that the water is warm and not hot because it can damage the pearls, especially if they are real. [substeps] A gentle dishwashing liquid or a laundry detergent meant for delicates works well for washing pearls.', '[substeps] You should use an all-natural dishwashing detergent if you use a dishwasher. Using a normal dishwasher detergent will ruin the pearls, which can make them hard to clean.', '[substeps] Pearls are delicate, so avoid adding too much, which could damage the pearls. [title] Apply a mixture of white vinegar and baking soda.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}