{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"87d5414a-293e-5a20-9ea4-24479de8dddc","task_key":"default--validation--37956","task_revision_id":"1","upstream_id":"37956","short_description":"HellaSwag validation 37956","config":"default","split":"validation","body":"{\"activity_label\":\"Cars & Other Vehicles\",\"ctx\":\"[header] How to get a commercial driver's license in maryland [title] Study the maryland commercial drivers license manual. [step] You will be tested on the information provided in the manual during the written test for your learner's permit. [substeps] Visit the mva website featured in the sources section of this article, then click on the \\\" forms \\\" link toward the top of the web page.\",\"endings\":\"['Enter the following information about the vehicle: year, model, mileage, registration status, insurance and identification information. [title] Fill out a form regarding the vehicle you are driving.', 'Click on the blue \\\" check your state \\\" link to review the forms section of the mva website. [title] Test at a national testing center (lds), as well as a state official testing center (ssn) in maryland.', 'Click on the link for \\\" commercial driver\\\\'s license manual form dl-151 \\\" within the commercial driver licensing forms section. You will have the option to read or print the manual from your computer.', 'Enter the information about you in the \\\" fill in the blank \\\" boxes at the top of the form. Have your permit either in the learner\\\\'s permit tab or under the \\\" contact us \\\" field provided.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}