{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"65a2282a-f4f7-57b9-8730-88ac8ad29f98","task_key":"default--validation--44033","task_revision_id":"1","upstream_id":"44033","short_description":"HellaSwag validation 44033","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to buy teacher supplies [title] Communicate with your supervisor. [step] A purchase order lets your school know what you're buying. Most schools require you to request a purchase order before shopping, especially if your school requires you to buy from specific stores.\",\"endings\":\"['Always, have a written record of the purchases you buy. [substeps] Your school should be able to supply you with a receipt of the purchase order.', \\\"Contact a teacher directly at or in your school and tell them you want the purchase order. [title] Talk to your school's principal, if there is one.\\\", \\\"Ask your school's bookkeeper if a purchase order is required before you start buying supplies. You'll need to write down each purchase on the order and return it to your school when you're finished buying your supplies.\\\", \\\"If you're unsure about what you can buy, ask your teacher. Depending on your school, you may also get help from a principal.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}