{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cb883481-e2fe-5cde-8ddd-53019a5cd9e0","task_key":"default--validation--682","task_revision_id":"1","upstream_id":"682","short_description":"HellaSwag validation 682","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to apply scene makeup as a preteen [title] Start with a clean face. [step] Before applying makeup, you should always have a freshly washed and moisturized face. This will keep breakouts to a minimum and your makeup will go on more easily and look better once it's on.\",\"endings\":\"[\\\"[substeps] If you don't have a clean face, you can use an acne cream instead. These can last until your eyes are shut, keeping your skin from producing acne.\\\", \\\"This doesn't mean that you need to wash it on. Your makeup must be applied immediately after your morning shower.\\\", '[substeps] Wash your face with soap and lukewarm water as a preteen. Use a makeup sponge or wash with warm water.', '[substeps] After you apply the moisturizer to clean skin, wait about 10 minutes before you start applying your makeup. This will give the moisturizer time to absorb completely into your skin.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}