{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"eb61540b-2052-56f3-8570-a6d7442b17ea","task_key":"default--validation--11004","task_revision_id":"1","upstream_id":"11004","short_description":"HellaSwag validation 11004","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to switch tabs in chrome [title] Switch to the next tab. [step] Press + to change to the next tab in the window. This will move you to the tab to the right of your current tab.\",\"endings\":\"['If you are already on the furthest right tab, this will send you to the one on the far left. This works on windows, mac, chromebook, or linux, but some operating systems have additional options : [substeps] You also have the option to use ctrl + pgdn.', '[substeps] Click link/click next to update. [title] Press + to switch tabs in the tab to the left.', 'You may also need to scroll down to find it. [title] Click on the tab to go back into browsing mode.', 'Move to the tab to the right of your current tab. [title] Click the \\\" pin up \\\" tab.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}