{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"94a0c4d9-73cc-5d1a-b287-373d3fb89dde","task_key":"default--validation--46185","task_revision_id":"1","upstream_id":"46185","short_description":"HellaSwag validation 46185","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to apply eyeshadow that lasts [title] Wash your face. [step] Starting with a clean and fresh face will make your makeup stand out more, help it last longer, and help prevent your makeup from clumping and flaking. Use your favorite gel or cream facial cleanser and lukewarm water.\",\"endings\":\"['Rub the cleanser in circular motions for 20 minutes before rinsing with warm water. [title] Apply your eyeliner to the crease of your eye.', '[substeps] Wet your face and pour some cleanser into your hands. Work the cleanser into a lather, and then gently massage it into your face for about 30 seconds using circular motions.', 'Tap your face with your fingers to get the excess off, then pat your face dry with a clean towel. [substeps] Do not splash water onto your face.', 'Pat your face dry with a clean towel. [substeps] An eyeshadow primer also works just as well.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}