{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9532a89b-ba41-5ad6-b4af-0f7be5a44b69","task_key":"default--validation--34002","task_revision_id":"1","upstream_id":"34002","short_description":"HellaSwag validation 34002","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to braid your own hair [title] Brush your hair. [step] This helps your braid stay smooth and neat. [substeps] Use any kind of brush or comb, brushes are better to smooth out your hair.\",\"endings\":\"[\\\"Brushing your hair before you start braiding will prevent knots from forming in your hair as your braid. Don't start braiding your hair when it is wet.\\\", 'Try not to use too much just in case any tangles are even. [title] Put a headband on over your head with a hair clip, to keep out fly-away hair.', 'Combing helps straighten and make your hair tastier. [title] Detangle your hair to make it easier to braid.', 'Comb all the excess hair out, so that all of your locks look smooth. Wet your hair after a shower so that it is not dripping wet.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}