{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"71f9179a-9094-51b4-b48c-29e888a68ccb","task_key":"default--validation--41176","task_revision_id":"1","upstream_id":"41176","short_description":"HellaSwag validation 41176","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make hair naturally wavy [title] Wash your hair. [step] Regardless of which of the below methods you choose, getting your hair wet first is an important step. Curls will look better in clean hair.\",\"endings\":\"['[substeps] Use lukewarm water to rinse your hair out, taking care to avoid touching the ends as you rinse. If using shampoo, you may want to wash it out first.', 'To get your hair wet, as soon as you head to the sink, squeeze it out. Only wet the ends of your hair, not the roots.', \\\"[substeps] Don't be surprised if some conditioner comes out just before you're ready to begin use. It works best when wet for hair that's already conditioned to reduce frizz, and if you've had it professionally styled since you last washed it.\\\", '[substeps] Use shampoo to make sure your hair is clean. When using conditioner, do not apply it to your scalp but instead focus on the ends of your hair.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}