{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"071370bd-2333-5045-bc6e-c40dd7265860","task_key":"default--validation--95","task_revision_id":"1","upstream_id":"95","short_description":"HellaSwag validation 95","config":"default","split":"validation","body":"{\"activity_label\":\"Travel\",\"ctx\":\"[header] How to get to the north pole [title] Book your flight. [step] The fastest and easiest way to get to the north pole, if you can afford it, is to fly. Flights to the north pole depart primarily from norway, but chartered flights are also available from canada fill out the paperwork and book your tickets.\",\"endings\":\"['[substeps] To fly from norway, you should expect to pay between ten and twelve thousand dollars. Visit the polar explorers website, open the \\\" expeditions \\\" tab and pick \\\" north pole flights.', 'Flights from denmark, iceland, and navy are also good options. You may want to check the fine print before booking your flight.', \\\"[substeps] If possible, book with someone who's also a flight instructor, as you'll need a good balance of military and civilian fun. [title] Enjoy ramps and trolleys to the north pole.\\\", \\\"English-language flights usually depart by the hour, though if it's january, you can opt to fly by the hour. [substeps] For example, norway-01:13 p.m.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}