{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"328d0576-e679-52fc-a6ed-55e1e7d65fb0","task_key":"default--validation--21823","task_revision_id":"1","upstream_id":"21823","short_description":"HellaSwag validation 21823","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to keep nail polish from drying out [title] Keep the cap on the bottle when you're not using the brush. [step] The number one cause of dried out polish is leaving the cap off of the bottle. A good general rule is to keep the cap on the bottle whenever you're not using the brush to apply polish.\",\"endings\":\"['[title] Allow to dry for five to ten minutes before opening the cap and buffing your nails. [step] If your nails are flaky, the moisture could cause the bristles to become gritty, and bacteria could spread into the polish.', \\\"If you stop using one color or switch to another, don't leave it open on the counter. Take a few seconds to screw the cap back on.\\\", \\\"If you're using the brush to style your nails, simply remove the cap and dip the brush in clear liquid nail polish remover. The rest of the bottle and the bottle itself can be dry using the cap removed.\\\", \\\"[substeps] Store the cap in the dishwasher or the refrigerator so it doesn't sit around for long. Reapply it whenever the cap starts to dry out.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}