{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d80e2e3b-a9bd-5933-9084-69c31ace200e","task_key":"default--validation--36177","task_revision_id":"1","upstream_id":"36177","short_description":"HellaSwag validation 36177","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to hide a big nose [title] Prep your face with cleanser and moisturizer. [step] Wash your face with a cleanser and pat it dry. Swipe a light toner over your skin before adding moisturizer.\",\"endings\":\"['[substeps] Starting with a clean face is an absolute necessity. Contouring is a makeup application technique that plays with the natural shadows of your face.', 'Blend it with your fingers and squeeze some moisturizer into your palm. Blend your free hand with a large fluffy brush until the moisturizer is evenly applied over the entire face.', 'Avoid products with bubbles, swirls, or clouds. You want to give your nose hair the attention it needs to grow healthy, and may also appreciate if you have a few hard-bud patches.', 'Wait a few minutes, and apply a cleansing product on top of your face. [substeps] You can find toners and soaps online that can be easily found at grocery stores, beauty supply stores, department stores, and pharmacies.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}