{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e2f65851-c78a-55e6-a834-891fd1ef9974","task_key":"default--validation--38496","task_revision_id":"1","upstream_id":"38496","short_description":"HellaSwag validation 38496","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to get an mmsi number [title] Make sure you're not required to obtain a ship station license. [step] If you're a recreational boater and aren't planning on visiting a foreign port, you're not required to register with the fcc. If you will travel to a foreign port, you'll need to apply for a ship station license with the fcc.\",\"endings\":\"[\\\"[substeps] You're allowed to travel in international waters without a ship station license. However, you can't travel to or communicate with a foreign port without the license.\\\", 'Some reputable port stations have almost as much as 24 hours notice at the door each day they let visitors board. [title] Realize that specific licensing requirements vary by port.', \\\"[substeps] Be aware that you won't be charged for your registration after you board an unfamiliar vessel. If you have the duty to register before you leave, the fcc will grant your license.\\\", \\\"That way, you can book within three weeks of your arrival for the license yourself. [substeps] You'll need to prove to the fcc that you've been a resident of the region for at least seven consecutive years.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}