{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bc47e557-530f-5ec2-a78c-867dc3d89e0c","task_key":"default--validation--48117","task_revision_id":"1","upstream_id":"48117","short_description":"HellaSwag validation 48117","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to apply pencil eyeliner [title] Consider what type of pencil eyeliner to use. [step] There are many different types of eyeliner that come in a pencil form, each of which has a different consistency and texture, application, and effect. [substeps] Powder-based pencils, sometimes called kohl, offer less intense color.\",\"endings\":\"['Single tbsp pencils, called sable pencil, are an alternative choice for wide eyes. Laser pencils, such as od, focus more harsh, dark tones.', 'They\\\\'re also a good option if you want to smudge the eyeliner for a \\\" smoky eye \\\" effect. Gel-or cream-based pencils offer simple and excellent application.', 'Stick with softer shades like red, cream, and champagne or sapphire to avoid blending many shades together. Red pencils have also been coated with synthetic pigments to create eye-catching colors.', \\\"Pearl-based pencils (and powder pencil pencils for shades of black) provide a more intense, subtle shade. [title] Look for a pencil that's loose and form-fitting.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}