{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"50933604-1baa-59b7-9dc9-71b2bb7c822e","task_key":"default--validation--34834--variant-2b0b46292835","task_revision_id":"1","upstream_id":"34834--variant-2b0b46292835","short_description":"HellaSwag validation 34834","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to apply eyeliner [title] Use a pencil liner for easy natural or smudged looks. [step] Pencils are easy to apply because you use them as you would any other pencil. You can lightly apply the pencil for a subtle, natural look, or you can smudge your pencil to create smoky or dramatic eyes.\",\"endings\":\"['Be sure to consider your chosen pencil liner size and shape when deciding what kind of look you want. For instance, if you have an oval face, choose a pencil that is slightly larger than your face in width.', 'Common pencils to use for this look are : [substeps] Lined brush [title] Apply eyeliner carefully. [step] Press the edges of your eyeliner down flat with your fingertip to smooth it out and make sure there is no excess liner on your eyelid.', \\\"It also won't run into your eyes like liquid liners can. [substeps] When using a pencil liner, you will need to occasionally sharpen or dull your pencil.\\\", \\\"[substeps] If you're going for a more natural or smokey look, apply eyeliner to the almond part of the eye. This part of the eye will be between the edge of the eyelid and eyebrow bones.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}