{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"955188f2-31c1-5911-b982-6baf5557f0f2","task_key":"default--validation--49295","task_revision_id":"1","upstream_id":"49295","short_description":"HellaSwag validation 49295","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to do a makeover with no makeup [title] Start with a clean surface. [step] Wash your face with a face wash or soap and warm water. Some soap can be drying on the more delicate skin of your face.\",\"endings\":\"['Choose a glass jar or sink with a bucket of warm water. Fill it with about a gallon of warm water and begin to swirl it around your face.', 'Only use ordinary hand soap to wash your face if you have no choice. Use a face cloth or flannel in small circular motions on your forehead and cheeks to help remove the dead skin cells of the day.', 'You could also use toner, instead. Let your face air dry as you normally would.', '[substeps] Cleanse your face with a gentle cleanser that comes in a bottle. Apply a cleanser of your favorite products to your face.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}