{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"044b0f95-6ee9-5566-8e90-6da8011184eb","task_key":"default--validation--42425","task_revision_id":"1","upstream_id":"42425","short_description":"HellaSwag validation 42425","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make a hair protectant spray [title] Pour the water into a spray bottle. [step] You can mix your hair protectant spray in the bottle that you plan to keep it in. Start by filling the bottle with 6 ounces (177 ml) of distilled or filtered water.\",\"endings\":\"[\\\"Add 6 ounces (100 ml) of pure distilled or filtered water. You'll need this water in every chemical form that you use and make sure that the bottle is filled with the ingredients that you'll need.\\\", '[substeps] Make sure that the water is at least 80% water or the spray will not achieve the same benefits as it would with distilled water. If your hair feels too wet and the curls are weak, consider running a wet hair dryer over it.', \\\"Because the water is denser than the oil, it's important to add it first so they mix properly. [substeps] You can use a plastic or glass spray bottle, but make sure that it holds 7 ounces (207 ml) or more.\\\", '[substeps] Make sure to use distilled water or filtered water. The carbonation in the water makes it more dangerous to use for hair products.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}