{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"29000e28-a115-57d5-800b-5d1503fcc6f7","task_key":"default--validation--37446","task_revision_id":"1","upstream_id":"37446","short_description":"HellaSwag validation 37446","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to be a man of the world [title] Practice good hygiene. [step] Personal hygiene is a basic part of showing others you're sophisticated. Bathe regularly, brush and floss your teeth, wear deodorant, and trim your nails.\",\"endings\":\"['They should be neat, odor-free, and sharp. However, loose nails can also hurt eyes.', \\\"[substeps] Choose clean, everyday clothes (and clothes that don't smell body odor) and wash your clothing at least twice per week. Wash your face, brush and floss once every other day, and use essential oil, even if it's not body-soft.\\\", 'If at all possible, have a dedicated figure-8-day spa day once a week. Use body spray and body oil both before and after you shower so your breath stays fresh and dewy.', 'Try to avoid leaving the house with wrinkled, dirty clothes or messy hair. [title] Build a button-down shirt collection.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}