{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5445704f-0bae-5ef7-bc06-9568d75aba10","task_key":"default--validation--50150","task_revision_id":"1","upstream_id":"50150","short_description":"HellaSwag validation 50150","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to put a streak of color in your hair [title] Choose a type of colored hair extension. [step] There are two main types of hair extensions: synthetic and real hair. Synthetic extensions are cheaper, and readily found in beauty supply stores and \\\" punk \\\" stores, such as hot topic or clairs.\",\"endings\":\"['[substeps] Synthetic extensions are considered very slick and straight with only a slight tug on the layers. Products containing synthetic hair (such as beauty supply, gel) work as well.', 'Dyed hair extensions: synthetic extensions use diluted oils in the hair to create the appearance of waves. They are available in many sizes and lengths, including perming, pompadour, and pilling.', 'Real hair extensions are more expensive, and can be found in salons and beauty-supply stores. Unlike synthetic extensions, they can be straightened and curled.', 'Real hair extensions come in this color, as well as awesome lengths and changes from light to dark. [substeps] Synthetic extensions come in a wide range of widths.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}