{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a2d1b756-fc8b-511b-b5f3-59197eb1e571","task_key":"default--validation--16808","task_revision_id":"1","upstream_id":"16808","short_description":"HellaSwag validation 16808","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to apply root concealer [title] Choose the right shade. [step] The range of shades available varies from brand to brand, but most of them offer around five options. These are typically light blonde, dark blonde, warm brown, brown and dark brown.\",\"endings\":\"['[substeps] A good rule of thumb is to choose just a color that matches your natural hair color. Start with a slightly darker shade (like blonde) and work your way up to lighter colors such as light brown, dark auburn or purples.', 'Find shades that are similar to your hair color and other skin tones to work on your complexion. Generally, black and warm brown shades work best for sun kissed skin.', 'Get the product that most closely matches your needs. [title] Dry and style your hair as usual.', 'Avoid applying your favorite shade and consider using more or less depending on your skin tone. If your skin is very tanned, then you might be able to find a shade that compliments your skin tone.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}