{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"703201dc-491f-560d-a0d1-aa3fb9a8e8af","task_key":"default--validation--27933","task_revision_id":"1","upstream_id":"27933","short_description":"HellaSwag validation 27933","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to get smokey eyes with makeup [title] Choose your colors. [step] Any color can be used to create a smoky eye, although you will need at least three shades of a similar hue. The classic smoky eye is created with black or gray, but bronze and brown are also commonly used.\",\"endings\":\"['[substeps] Green eyes look particularly good with a gray and plum smoky eye, while blue eyes pop with gold or copper and brown eyes go great with shades of navy and gray. You should choose three shades of each color: a light, creamy color; a medium base color; and a dark, smoky color.', 'Black is usually used, but you will need to consider depending on the mood you want to convey. If you want a smoky eye, a black lid is most attractive, and so is black eye liner.', 'Your choice of colors depends on what face you have to look at. [substeps] Browns with cool blues work great as a base shade for smokey eyes.', '[substeps] If you are looking to make smoky eyes on people else, try buying them in neutral shades: ivory, olive, taupe and light beige. If you are going to make these eyes on another person, avoid bright colors, like purple or yellow.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}