{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"781f4581-368c-51b2-9ca1-59d400b142f8","task_key":"default--validation--25547","task_revision_id":"1","upstream_id":"25547","short_description":"HellaSwag validation 25547","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to dye hair blue [title] Start with a clarifying shampoo. [step] Using a clarifying shampoo can help to remove buildup from your hair and make it easier for you to dye it. It can also help to remove leftover dye from the last time you dyed your hair.\",\"endings\":\"[\\\"Look for clarifying shampoo that contains aluminum sulfates and which are made especially for hair that isn't dyed. Remember that green hair dye is very dark blue, so it's really best to use a shampoo made from other green products.\\\", '[substeps] Clarifying shampoos are also generally safe for use as their names imply. [title] Rinse your hair with cool water.', \\\"[substeps] Look for clarifying shampoos at beauty supply stores like lowe's and beauty sugar's. They can generally be found in various colors.\\\", 'You can find a clarifying shampoo in beauty supply store and some drug stores. [substeps] Follow the instructions on the package for the clarifying shampoo.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}