{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"94bfc650-a886-5ccd-b73c-f6031dd76bf9","task_key":"default--validation--15352","task_revision_id":"1","upstream_id":"15352","short_description":"HellaSwag validation 15352","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make brown eyes stand out [title] Try gold eye shadow. [step] Gold is a great way to add some pop to your brown eyes. Use gold eyeshadow on your eyelids to highlight the gold specks in your eyes.\",\"endings\":\"['Try getting to the part of your eyelids that comprises most of your brown eyes, then consider adding black. Continue to add gray eyeshadow on those areas that you would like to highlight.', 'Apply a matte gold shadow or a metallic gold shadow for some real pop. [substeps] You can also try other warm metallic shades like copper and bronze on your eyes to make them stand out.', 'Gray eye shadow colors like light gray, brown, tan, and neon blue work best. [substeps] You can also use some artificial eyeliner to darken the eyes.', '[substeps] Consider using almond-shaped eyeshadow for blue eyes. A shimmery foundation works better for golden eyes.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}