{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c68933e6-15d3-519f-801f-d31933a18774","task_key":"default--validation--44062","task_revision_id":"1","upstream_id":"44062","short_description":"HellaSwag validation 44062","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to establish an effective skincare routine [title] Remove makeup. [step] If you wear makeup, you should remove it before you go to bed. Some cleansers claim to remove makeup for you, but these may not get rid of all makeup.\",\"endings\":\"['You may want to have an extra makeup remover on hand. Use it before you cleanse your face.', 'Makeup can wash away after you shower, so take off all makeup you will be putting on before bed. Your makeup should rest on your face instead of covering it on stage.', \\\"You'll need to remove makeup before you sleep so you can properly prepare for bed. Try using your regular lotion or facial wash when you wake up to keep it off.\\\", 'Get out a washcloth, cotton ball, or a makeup sponge and squirt some cleansing oil into them. [substeps] You may want to wait until after you begin sleeping to remove makeup.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}