{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1502a89a-a2c6-512d-aa2d-07ad83f755cd","task_key":"default--validation--18416","task_revision_id":"1","upstream_id":"18416","short_description":"HellaSwag validation 18416","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to change your eye color [title] Understand how makeup can alter eye color. [step] You cannot use eyeshadow to make blue eyes look brown, and vice versa. You can, however, use eyeshadow to enhance the color you already have.\",\"endings\":\"['This can assist with eye color even when wearing different eyeshadow shades. [substeps] Eyeshadow works slightly differently in blue eyes.', 'Try it on and see if your eyes really look mismatched. Then, make sure you choose a color that matches your skin tone (pretty much any face color).', 'Depending on the eyeshadow color you use, you may be able to make your eyes appear brighter, duller, or more pale in color. Some eye colors, such as hazel and grey, may be able to pick up tints from certain eyeshadow colors.', 'When you wake up in the morning, you might notice your eyes looking more washed out than they did before you went to bed. [substeps] Some eyeshadow blends irritate your eyes, making it hard to apply neutral makeup without blinking.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}