{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1227197c-ecf5-5390-9172-6aacd5b42158","task_key":"default--validation--7875","task_revision_id":"1","upstream_id":"7875","short_description":"HellaSwag validation 7875","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to manage your beard [title] Eat a healthy diet. [step] A healthy diet generally promotes healthy skin and hair, and this includes your beard. [title] Don't succumb to the itch.\",\"endings\":\"[\\\"[step] Beards often remind people of their beards once they realize they've dislodged their actual beard. You may not realize it, but bits of beard hair will develop into small blackheads to the shape of beard hairs if an adult is not careful.\\\", '[step] Itching may persist in new beard growths at first, but it will go away on its own by the end of the day. [title] Wear a beard with a rigid center.', \\\"[step] When you're trying to maintain your beard, the itch is the worst thing you can do! It is likely to last even for a few days or weeks before the itch disappears. [title] Write down everything you're thinking about about in your head.\\\", \\\"[step] As the hair transitions from stubble to a full-fledged beard, you are likely to encounter several weeks of itch. The itching is unpleasant, but don't despair because it will go away.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}