{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3e0e9c40-f67f-5f9d-a011-bc1924929f6b","task_key":"default--validation--33246","task_revision_id":"1","upstream_id":"33246","short_description":"HellaSwag validation 33246","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to groom yourself [title] Wash your face twice per day. [step] In the morning, a water rinse will suffice for most skin types. Use a mild, gentle cleanser and lukewarm to wash your face every evening.\",\"endings\":\"['Do not use any soaps or cleansers with moisturizing agents or preservatives. [substeps] You want your skin relatively fresh after washing.', 'Use a facial cleanser that uses salicylic acid for sensitive skin. For ears : [substeps] Wash your ears twice.', 'Gently pat your face dry with a clean washcloth. Avoid scrubbing your face dry, which can damage your skin.', 'Take your time with this cleanser, and remove every layer of dead skin. Start at your jawline and then work your way down to your nape.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}