{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1ff78561-9432-51df-9c9d-4144820cb1b5","task_key":"default--validation--41682","task_revision_id":"1","upstream_id":"41682","short_description":"HellaSwag validation 41682","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to dress when you're a teacher [title] Find out if your school has a set dress code. [step] Some schools will have a standardized dress code for teachers that outlines what you can and cannot wear. Speak to the school principal or a school administrator to get a copy of the official dress code for teachers.\",\"endings\":\"['This will let you know if there are any consequences if you disobey this code. [substeps] However, there are some unique rules that state different etiquette for school dress.', '[substeps] Even if there is no official dress code, most schools will provide a list of suggestions for teachers on what is appropriate and inappropriate to wear. [title] Invest in staple items like comfortable dress pants and blazers.', '[substeps] Many schools have \\\" no underwear \\\" lines where you may want to remove underwear so you can change into something. Some schools do not allow underwear in uniform, and some do not permit it at all.', 'It may take a while to get a copy, especially around the holidays. If you do not know what you want to wear, do some research to find out.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}