{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"25faf72a-ed1f-5746-b0cc-7767c7ee2c38","task_key":"default--validation--1966","task_revision_id":"1","upstream_id":"1966","short_description":"HellaSwag validation 1966","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wear black lipstick [title] Exfoliate and moisturize your lips. [step] Black lipstick starts to look streaky and flaky more quickly than other colors. To prevent this from happening, it's important to prep your lips before you apply it.\",\"endings\":\"['[substeps] Exfoliate for at least 15 minutes each. Follow up with a light moisturizer to replenish moisture.', \\\"Make sure you remove all skin flakes and moisturize your lips so they're soft and supple. If your lips are chapped, you might want to save the black lipstick for another time.\\\", 'To do this : [substeps] Apply your moisturizer. Read the package to see how long it will take to dry up.', 'Take a piece of cotton print paper and lay it across your lips, brushing out the surface until you\\\\'ve created perfect lines on your lips. [substeps] When you seal your lips around the paper, you\\\\'ll have a kind of \\\" canvas \\\" over your lips.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}