{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"49a3050b-5360-56dd-95cb-714d4b33e583","task_key":"default--validation--513","task_revision_id":"1","upstream_id":"513","short_description":"HellaSwag validation 513","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to survive through a long car journey with teenagers [title] Clean the car out. [step] The last thing you want is to be sitting on toys and/or sweet wrappers. [title] Make a carry on bag that includes items for each person in the car.\",\"endings\":\"['[title] Give on the plastic bag for each individual but keep on the bag so everyone can just hang their stuff under the seats and in their own bags. [title] Make sure that all the important things are stored in your carry ons.', '[step] You could let this space for your kids to sleep on! [title] Check in with the website for your area. [step] Get a phone and text list that includes your mobile phone details and a list of things to pack for each person in the car.', '[step] They need something to carry throughout the trip. In order to make sure everything is packed, have this bag open at the last minute when you have time to do anything else.', '[step] In the bag put some books/comic books, magazines, nintendo ds. , psp and any other portable games.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}