{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9bb4d1d5-8919-5569-98c3-8c30524bb117","task_key":"default--validation--44248","task_revision_id":"1","upstream_id":"44248","short_description":"HellaSwag validation 44248","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wear colored jeans [title] Set the tone with your jean color. [step] Wearing neutrals with your colored jeans allows you to wear jeans in both bright and subtle shades, so the color you choose will be what sets the tone of your outfit. [substeps] For the most subtle look possible, choose a jean in a pastel hue or a deep tone.\",\"endings\":\"['For instance, you could go for a light mint green jean or a deep olive green jean. For a bolder look, go with a bright color.', 'For a neutral look, choose a gray tank top or a gray hoodie. Wear your jean color elsewhere when selecting denim pants.', \\\"Most neutrals today will tie into more sophisticated styles, but this isn't always possible. For example, wearing a striped top will clash dark colors well; if you opt for a striped top, you'll be more closely matched with this style.\\\", 'Pick out the color you want to complement the tone of the denim. Choose bright, vibrant colors, like pink and blue for spring and green for summer and neon for winter.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}