{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4ef411c7-e138-5682-9e2f-5bcdbbe4bc2b","task_key":"default--validation--11896","task_revision_id":"1","upstream_id":"11896","short_description":"HellaSwag validation 11896","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to do a cut crease [title] Start with a bare face. [step] This way, you won't have to re-apply foundation if you make a mistake, and you'll avoid smudging your foundation as you work on your eyes. Start by washing your face with your usual cleanser and applying moisturizer.\",\"endings\":\"[\\\"This will keep your face smooth and clear. [substeps] You don't have to create a huge pattern here-you can just leave the first line smooth.\\\", \\\"[substeps] To apply mascara, you will be adding a heavy, clean eyelash from above your eye. The mascara will increase your eyelashes' visibility under the red eye.\\\", \\\"Then, lightly work the concealer into your hairline, from the bridge of your nose to the outer corners of your eyes. [substeps] You should feel no pain while applying the concealer at this point because you'll be painting over it more later.\\\", '[title] Cover your eyelids with a light layer of eyeshadow primer. [step] This will help the color last and blend more naturally.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}