{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b2dbbc07-3137-502c-99eb-e0a81411623f","task_key":"default--validation--4420","task_revision_id":"1","upstream_id":"4420","short_description":"HellaSwag validation 4420","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to ace a pageant interview [title] Dress appropriately. [step] Your outfit should be formal, yet business-like. Unless otherwise indicated, a business suit will appear to stiff.\",\"endings\":\"['Show up to your appointments smelling fresh and clean. [title] Brush your teeth and make sure your hair is neatly combed.', 'A nice, professional-style dress or a silk blouse with a pencil skirt will do just fine. Be sure that whatever you wear touches or goes past your knees, as anything shorter may make you seem \\\" floozy \\\".', \\\"If you're wearing glasses, as long as your glasses fit properly and won't slide off, you should be okay. A tie and some dress pants will work, though dinner or business casual aren't standard.\\\", 'Business casual is often better for pageant interviews. [substeps] When dressed formally, you can look as professional as possible.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}