{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d83280cc-66ff-57ff-9047-a2b84ef9d3a7","task_key":"default--validation--47838","task_revision_id":"1","upstream_id":"47838","short_description":"HellaSwag validation 47838","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make lipstick [title] Get lipstick base supplies. [step] The lipstick base makes up the majority of the substance, and you customize it by adding pigments. The ingredients you use to make your lipstick base can be tweaked to make the lipstick more glossy, matte, or balm-like.\",\"endings\":\"[\\\"Here's what you need : [substeps] 1 teaspoon (5 ml) beeswax or beeswax beads. You can find beeswax at craft supply stores.\\\", \\\"Store the bases with plastic bags in an airtight container in your refrigerator if you don't want them in your purse. Don't forget to include fabric balms to match the color of your lipstick base.\\\", 'You can use any color you want; reds, pinks, and other shades with matte bases compliment the lipstick more. You may also want to consider colors like black or gray, for example.', \\\"[substeps] Make sure you have these ingredients on hand. If you can't find any, use latex paint with your lipstick base.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}