{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7817a839-8745-5a35-879d-84fe59583744","task_key":"default--validation--39085--variant-22ce9b50a63a","task_revision_id":"1","upstream_id":"39085--variant-22ce9b50a63a","short_description":"HellaSwag validation 39085","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make your own lip color [title] Sort your lip products by color. [step] When you're mixing lip products to create a custom color, it helps to separate them according to color. That will make it easier when you're trying to tweak the color exactly to your specifications.\",\"endings\":\"['[substeps] For example, you might make a pile of red lip products, one for pink, one for nude, and so on. You may also further sort your lip products by type.', 'Examine your lip products: what do you want to do with the lipstick, moisturizer, and eyeshadow? [substeps] For a reddish lip color, they should both be shades that complement each other colorfully. For a shimmery lip color, they can both fall into the same color palette.', \\\"[substeps] If you want to create mouth colors that match the lipstick, such as pink or blue lip products, start by sorting items by color. If you're trying to add natural beauty to lips, such as liner, you'll work on sorting, then work from there to mixing.\\\", 'For instance, if you want a light tone but no darker, sort the pieces of lip liner in favor of the lighter ones. [substeps] This helps you find all the parts in your makeup in the same spot.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}