{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"276ba18a-c682-5b3c-98db-c9c40bd0189b","task_key":"default--validation--37632","task_revision_id":"1","upstream_id":"37632","short_description":"HellaSwag validation 37632","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to pack a bag for before school sports practice (for girls ) [title] Find a bag. [step] Hopefully, you have a locker at your school. You may like to use a medium-sized drawstring bag from puma, any brand you like will work.\",\"endings\":\"['[substeps] You should pack one in case, and also if you go to the gym, try to see if you can get any sports bags. Sometimes this is not possible, but parents occasionally say they are about to have a hockey game so you can inspect the bag a little more before the game.', 'You can also use a small duffel bag. Try brands such as puma, nike, adidas or use a perfectly good unbranded bag [title] Pack your bag with your sports clothes.', \\\"If you're really embarrassed, right? You can take it with you wherever you go. If your school doesn't have any lockers, ask your parents or principal for a bag.\\\", \\\"Look to make sure your bag has plastic flaps or plastic compartments to stop things from coming through, so you don't have to take everything out. [title] Cover your dorm with an old comforter.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}