{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f687ee84-85c6-5160-ae36-523dbc1bf817","task_key":"default--validation--10204","task_revision_id":"1","upstream_id":"10204","short_description":"HellaSwag validation 10204","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to push back your cuticles [title] Prepare a bowl of warm water. [step] It shouldn't be too hot, or too cold. Soaking your nails in warm water makes the cuticles soft and pliable.\",\"endings\":\"['Let them soak for several minutes before moving on to the next step. [substeps] You can also put them in ice water to help soften them slightly.', \\\"When you push them back, they'll lift easily from your nails instead of sticking. You can also push them back after taking a warm shower, while your hands are still warm and moist.\\\", 'This will help to loosen them, while making them easier to push. [title] Run a thin fingernail under water and tap it against your cuticles.', 'Therefore, you may want to use an epsom salt bath before attempting this method; it is not recommended for calluses. [title] Apply lukewarm or warm water to the cuticles.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}