{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"62de143f-7a28-528f-b86b-b1384c5d0c7d","task_key":"default--validation--15542","task_revision_id":"1","upstream_id":"15542","short_description":"HellaSwag validation 15542","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to rename a drive [title] Access window's disk management feature. [step] Open the control panel from the start menu and, if necessary, switch to the default view by clicking \\\" control panel home \\\" in the upper-left corner. [substeps] Select the \\\" performance and maintenance \\\" option and click the link to the \\\" administrative tools.\",\"endings\":\"['\\\" if your system requirements for the drive have changed, you can formatted your drive to some perspective. You can preview the new processing settings in the \\\" performance \\\" section of the main control panel, and enjoy the free trial.', '\\\" look for a running driver firewall installation window, then find the \\\" run \\\" tab. Click \\\" run \\\" in the windows installation window to install the new driver.', '\\\" [title] Access the user user profile summary window and explore the user\\\\'s profile profile or profile. [step] Before you delete a user\\\\'s profile profile or profile, navigate through the user profiles or profiles section at the top of the window.', '\\\" double-click the \\\" computer management option \\\" icon and select \\\" disk management \\\" from the options listed in the panel on the left. [title] Locate the volume, drive or partition to be renamed from the list of available options.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}