{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b4375d6c-71d4-59ed-b913-72ea389c758a","task_key":"default--validation--35851","task_revision_id":"1","upstream_id":"35851","short_description":"HellaSwag validation 35851","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to dye your hair with eyeshadow [title] Find suitable eyeshadow. [step] Choose old eyeshadow or some from the dollar store. It's probably best to not use your good quality, expensive makeup.\",\"endings\":\"['Use one that matches your skin tone and that suits your coloring. [title] Fill your eyeshadow container with the eyeshadow you want.', '[title] Crush the eyeshadow into a fine powder. [step] Use a fork, end of a butter knife, or similar to help you mash it up in a bowl.', \\\"If it gets ruined at some point, your outfit won't be worth squinting at. [substeps] Don't use black eyeshadow because it is more intense and illuminates.\\\", \\\"However, you can use some eye shadow in natural shades like dark green or deep cranberry. Just make sure it's muted so you can find it later.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}