{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f5d36977-3f38-5361-9878-7ccae9e89ecb","task_key":"default--validation--31654--variant-457d39903c67","task_revision_id":"1","upstream_id":"31654--variant-457d39903c67","short_description":"HellaSwag validation 31654","config":"default","split":"validation","body":"{\"activity_label\":\"Travel\",\"ctx\":\"[header] How to vacation in puerto rico [title] Think about your priorities. [step] As a relatively populous island nation with diverse regions, puerto rico offers a variety of options for travelers. Before planning the specifics of your trip, you need to decide on the purpose of your trip.\",\"endings\":\"['You can opt to have it just one day or only an hour over a couple of days to keep up your travel time and avoid crowds. [substeps] Spend plenty of time trainings with your fellow travelers.', 'Try out the following options to help you get started : [substeps] One of the great things about puerto rico is its unique sights. Since the average tourist season passes between june and november, you may want to show up with your tourist ticket and some hotels nearby.', 'Think about if your trip to puerto rico will be : [substeps] A relaxing vacation. An adventure, full of hiking, urban exploration, and activities like snorkeling.', 'Sometimes traveling to puerto rico is due to a desire to live on the islands or other asian culture. Other times, traveling from a city city or large city to a short, red-colored city in puerto rico can lead to de repercussions or a situation where puerto rico would be intolerable.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}