{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e2eefd6d-b16d-59c4-8f5a-6f243109b83c","task_key":"default--validation--49063","task_revision_id":"1","upstream_id":"49063","short_description":"HellaSwag validation 49063","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to apply metallic eyeshadow [title] Add a primer first. [step] A primer is just a light makeup you apply to your eyelids before eye shadow. It can help keep your eye shadow in place, which is good for metallic eye shadows since they are more likely to come off.\",\"endings\":\"['Plus, it can really make the color and shimmer pop. [substeps] Apply the primer across the the top of your eyelid, going all the way up to your eyebrow.', 'Take a primer brush and sponge some primer onto your fingertip. Gently dab the primer onto your eyelids.', 'Using a primer first will also make a primer easier to apply, allowing for easier blending. [title] Dry your eyelid with the primer over the top.', 'This can also help prevent primer from drying on your eyelids and causing them to wrinkle. [substeps] Primer is used to help in check eye colors for photographs and is about the last thing to bring out a metallic eye shadow.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}