{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"660870ba-1fac-5e3a-96bb-ac6131f48241","task_key":"default--validation--26351","task_revision_id":"1","upstream_id":"26351","short_description":"HellaSwag validation 26351","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to get boring homework done [title] Organize your supplies. [step] Before you can tackle any homework, you need to make sure that you have everything you need to finish the job. You can avoid needing to get up every few minutes by gathering all your supplies first.\",\"endings\":\"['This also lets you leave a little time to do a loose break and to begin your work. [substeps] Get as many pictures and books as you need to put in your backpack.', 'Take your binder, two books, your pencils, and a calculator and divide the papers into threes. Place these into one room, dividing the remaining papers in a safe place.', '[substeps] Take out all of your homework from your school bag, including your agenda, if you use one. Go through each homework task and determine what supplies you will need.', \\\"[substeps] Going through your backpack and placing everything on top will also help move things around. Being organized is particularly important if you're working on a deadline.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}