{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"598acb43-fd8b-55d0-ac4b-77b34191c9d7","task_key":"default--validation--2632","task_revision_id":"1","upstream_id":"2632","short_description":"HellaSwag validation 2632","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to dye your hair light brown [title] Lighten with bleach first if you have black or chemically treated hair. [step] If you fall into one of these categories, there's no getting around pre-lightening with bleach before you move forward. Purchase a vol 20 or 30 bleach kit from a beauty supply store and follow the instructions that come with it.\",\"endings\":\"['[title] Buy dye goggles that are waterproof and have side glasses. [step] Waterproof goggles are important for hair that ends up in an intense bleach hue.', 'Most dyes will contain lightening toners and peroxide, but some contain synthetic ingredients, as well. Be aware that having these ingredients added at once could alter your hair color, so be prepared.', 'Your new hair color will not show up if you proceed without bleaching the current color out first. [substeps] If you have red, dark or medium brown hair, use vol 20 developer.', 'Mix and pour on all the bleach you desire. Roll it into a ponytail and secure it with a hair tie before applying to your hair.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}