{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"77558ff9-11bf-5fe3-970e-f11411942ec2","task_key":"default--validation--46344","task_revision_id":"1","upstream_id":"46344","short_description":"HellaSwag validation 46344","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to cover skin flaws with makeup [title] Apply primer to freshly washed skin using your fingertips or a brush. [step] A primer is an important step because it means that your makeup will go on smoothly and will stick to your skin all day. Allow the primer to sit on your skin for a few minutes before moving to the next step.\",\"endings\":\"['[title] Apply concealer to pimples and scars. [step] Cover the sides of the blemish and/or dark spots completely by creating an \\\" x \\\" movement with the formula (preferably one a shade or two lighter than your skin tone).', '[title] Lightly dab primer onto your foundation. [step] It should be applied evenly across all the area of your face except for your hairline.', 'The primer should be solid enough to stick to your skin, but not too solid that you cannot easily remove it. [substeps] An opaque primer paint is a great choice, as it can be applied through a thin film without adhering to your skin.', \\\"You will have to squeeze water into the primer all day to keep it soft. Apply the primer to your palm if you don't have a brush for this first step.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}