{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a9b1e935-0060-5e9a-8a9a-bdd1b56a279d","task_key":"default--validation--17458","task_revision_id":"1","upstream_id":"17458","short_description":"HellaSwag validation 17458","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to straighten curly and thick hair [title] Wash your hair with shampoo and conditioner to rinse away the dirt. [step] Use a gentle shampoo and conditioner as flat-ironing hair may cause damage to your hair. Shampooing and conditioning will remove unwanted oil and product build-up from your hair.\",\"endings\":\"['[title] Scrape out dry, damaged, split ends using a comb or a rat tail comb. [step] Coarse, split ends can be tight on a rat tail comb.', '[substeps] If you\\\\'re going to wash your hair with soap, be sure to pay special attention to your hair\\\\'s natural oil content. The bottle should say \\\" baby shampoo \\\" and \\\" natural \\\" on the label.', '[substeps] You can skip this step if your hair has been bleached recently. [title] Apply a serum or oil to your hair, starting at the roots and working downward.', '[substeps] Look for shampoo and conditioner products that are sulfate-free and jam packed with glycerin. Glycerin helps reduce frizz as it penetrates hair follicles, hydrating hair from the inside out.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}