{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e878af93-677e-5665-9ea0-f4383ebb7daf","task_key":"default--validation--34599","task_revision_id":"1","upstream_id":"34599","short_description":"HellaSwag validation 34599","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to lowlight gray hair [title] Choose a hair dye that is 1 to 2 shades darker than your current color. [step] If you go too dark, your hair will look stark, streaky, and unnatural. The darker pieces of hair might overshadow the gray hair and make it look as though your hair is currently graying.\",\"endings\":\"['[title] Select a hair dye that goes at least 1.5 shades darker than your current hair color. [step] Black hair dye is just like gray hair color, except it comes in a variety of shades.', 'For darker hair you could use just a little lighter dye (brown, warm blue, etc. ) or one to two shades darker (purple, blonde, etc.', '[title] Follow the instructions that came with your dye. [step] Some dyes require you to read the directions on the bottle before mixing, others will require you to monitor the instructions on the bottle.', '[substeps] If your hair is more of a brownish-gray, stick with dark-brown. If you have dark-gray hair, consider going with black lowlights.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}