{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"56a45eae-9e57-5d45-91e5-26ac54aa84a0","task_key":"default--validation--7283","task_revision_id":"1","upstream_id":"7283","short_description":"HellaSwag validation 7283","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to apply gradual tanner [title] Choose your product. [step] Most big skincare brands now make a gradual tanner, so you have a lot of choice. The main products are dove summer body, johnson's holiday skin, l'oreal nutrisummer, no7 naturally sunkissed gradual body tan, garnier summerbody and palmer's natural bronze.\",\"endings\":\"['[title] Buy a primer or spray bottle, and keep the primer or spray bottle on hand. [step] This means there will be primer as your skin adjusts to tanning, such as when your skin is dry.', '[substeps] These products can be purchased from a tanning website or from suppliers. Be sure to find a product without quotes or links to information.', 'These products are found in several different good brands. Honey is the super popular product in places, including drugstores, grocery stores, most drug stores, and every drugstore.', 'Most products come in two versions, one for pale skin and one for dark skin, so choose the right one for your skin tone. Some gradual tanners also come with a hint of shimmer, so are ideal for taking on holiday.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}