{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7a2f889e-4f74-509c-8d3b-6da1a4332fed","task_key":"default--validation--12676","task_revision_id":"1","upstream_id":"12676","short_description":"HellaSwag validation 12676","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to do gradient smokey eye makeup [title] Pick a base color. [step] One of the more popular base colors for a smokey eye look is black or gray. You may go for black or gray as your base if you want a more traditional smokey eye for a night out.\",\"endings\":\"['Black or gray also be a good option if you are planning on wearing a colorful outfit and want your eye makeup to be more neutral and dark. Another option is to go for a base color that is more colorful, such as blue, purple, green, or pink.', 'Alternatively, you may mix it up with other colors so that your smoky eye looks the same. [title] Mix dark eyeliner with dark eyeliner.', 'Or, you may choose an instantly interesting shade for an eye the exact same color as your shadow color. Match the color of your base color to your shadow color to give it dimension.', '[substeps] Most earth tones are deep and muted. For a darker smokey eye, try pink, black, or purple.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}