{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6882806e-e11b-58ae-8361-4413a91d84ab","task_key":"default--validation--31225","task_revision_id":"1","upstream_id":"31225","short_description":"HellaSwag validation 31225","config":"default","split":"validation","body":"{\"activity_label\":\"Sports and Fitness\",\"ctx\":\"[header] How to get a gun license in massachusetts [title] Find a licensed firearms safety course. [step] In order to apply for a gun license, you will need to complete a firearms safety course. You can find one in several ways : [substeps] Visit the massachusetts executive office of public safety and security, at www.mass. Gov/eopss.\",\"endings\":\"['Follow the link to \\\" firearms registration and laws \\\" in the top right corner of the page. Select \\\" approved basic firearms safety courses \\\" to access a complete list of courses approved by the state of massachusetts.', \\\"If you are a resident or a non-resident, you may use the website srssionenforcs.gov to locate a local or state firearms training program in your area. You may also visit the massachusetts dmv's website, at www.srssionmiddle.\\\", 'During the course, you will learn about firearms and how to get an official license. If you do not attend the classes this year, you cannot purchase an official gun license.', 'Select the county or state where you live by pressing enter to begin the course. A detailed account of your history with the gun and where you first purchased it will be displayed on your license.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}