{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f1db758e-b93c-5845-8da5-04e981b3a0fc","task_key":"default--validation--17645","task_revision_id":"1","upstream_id":"17645","short_description":"HellaSwag validation 17645","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to apply highlight and lowlight foils to hair [title] Pick your lowlight colors. [step] Choose hair colors one to two shades darker than natural color for natural results. For more dramatic results, go for shades two to four shades darker.\",\"endings\":\"[\\\"Otherwise, don't apply your lowlight on hair colors that won't react with the natural color. [substeps] If you want your highlight to remain light, choose bold reds and lowlight purples instead.\\\", 'If this is your first time dyeing your hair, you may want to try using temporary or semi-permanent dye rather than permanent dye. [substeps] Temporary dyes tend to wash out after one shampoo.', '[substeps] If you want to highlight and lowlight the top layer, apply solid cream or cream at the end. Apply the powder to get a thick, glossy finish.', '[title] Next, dip a set of pins into makeup remover. [step] Apply makeup remover to your hair.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}