{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"db66068f-ee43-5ebf-ac7d-641b732e6255","task_key":"default--validation--2992","task_revision_id":"1","upstream_id":"2992","short_description":"HellaSwag validation 2992","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to do makeup for hazel eyes [title] Choose eyeshadow in a shade of brown or gold. [step] Using an earthy color will accent the brown in your eyes, causing them to look deeper and darker. Look for a brown palette with a range of colors so you can experiment with hues that bring out the best in your eye color.\",\"endings\":\"['[substeps] For daytime wear, look for neutral, sandy or milk chocolate tones that accent your eyes without standing out too much. For nighttime wear, look for dark chocolate or shimmery gold tones that draw attention to your eyes.', '[title] Use eyeshadow that matches your actual eye color. [step] Hazel eyes have a yellow tint to them.', '[substeps] A palette of shaded eye shadow-particularly earth tones-is best because it mimics the lashes, highlighting them well throughout the day. A palette with bright colors will work best.', 'Go with a solid brown shade for hazel eyes so you can match them better. [substeps] Light brown eyeshadow works well for hazel eyes because it will show as much color on your lashes as it will on your cheeks.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}