{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"305d8e35-08c1-532a-b3fa-6c7fff10c5b7","task_key":"default--validation--45655","task_revision_id":"1","upstream_id":"45655","short_description":"HellaSwag validation 45655","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wear berry lipstick [title] Decide how dark and opaque you want the lipstick to be. [step] There are a wide variety of berry lipsticks on the market, which run the spectrum from light to dark and highly opaque to sheer. Determine how intense you want your berry lip color to be, so you can choose the appropriate lipstick.\",\"endings\":\"[\\\"[substeps] Dark, opaque berry lipstick can look very harsh if you have fair skin. That doesn't mean you can't wear a dark berry lipstick if you have pale skin.\\\", '[title] Choose lipstick you are okay with using. [step] The quality of your lipstick can have a big impact on your hair color.', \\\"[title] Lightly fill a funnel with lip gloss. [step] Once you've selected the color of lipstick you want to use, carefully fill a funnel with lip gloss.\\\", 'Some popular berry lipstick brands include : [substeps] Barbacoa black petroleum jelly [title] Find a one-half inch scoop to stick your lip with. [step] Tip your head back so you can see the object in your hand and your lip.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}