{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6a4a3a0e-431e-5bc2-9b37-44dd05da5c70","task_key":"default--validation--7912","task_revision_id":"1","upstream_id":"7912","short_description":"HellaSwag validation 7912","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to visit sagrada familia [title] Research travel options. [step] Since barcelona is the second largest city in spain, getting there should not be too difficult. Depending on how far you are traveling, you can fly into the barcelona-el prat airport (bcn) or take a train from around spain, france or even london.\",\"endings\":\"['[substeps] If coming from nearby, la sagrada familia is located at caller de mallorca , 401. There are plenty of lodging options available nearby as well, including hostels, apartments, and regular hotels.', \\\"It usually takes around 20 minutes to get there, so don't worry about waiting much longer. It's important to realize that barcelona won't be big enough to accommodate five to twenty people.\\\", '[substeps] Explore some of the tourist destinations located in the city for a reference while planning your trip. Fly in advance to be there, because barcelona is such a very active tourist.', \\\"[title] Drive for about 2.2 hours to get to barcelona. [step] The driving can be expensive, so don't waste money in small amounts by arriving so early that you couldn't just go directly to the airport.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}