{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"68c56122-55fd-5df3-89b4-09dffd5f48bb","task_key":"default--validation--27585","task_revision_id":"1","upstream_id":"27585","short_description":"HellaSwag validation 27585","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to do tiger eye hair [title] Purchase your hair dye. [step] Take a trip to your local drug store or beauty supply store. Depending on the look you're going for, you may want to keep it simple and just choose one color, or you may want to buy four.\",\"endings\":\"['Your hair will stick out more if you use a thicker dye, such as a mousse or gel. [title] Pour 3 ounces of red wine into your bowl.', \\\"[substeps] In the tattoo artist's shop you should find several different colors and strips, and apply those to your hair. Make sure the colors match what you want to do.\\\", \\\"It's totally up to you! You can buy a blonde highlighting kit that will lighten pieces of your brown hair, auburn dye, gorgeous golden hues, soft brown dyes-whichever dye you think will look good in your hair. The darker your hair is, the less of an effect you'll notice.\\\", '[substeps] If you want to dye your hair yourself, make sure to use the formula before you apply the dye. Gels are sometimes recommended but are highly expensive, and can be difficult to find at grocery stores.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}