{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"81380993-d07b-5c01-9782-69784a7696cc","task_key":"default--validation--23512","task_revision_id":"1","upstream_id":"23512","short_description":"HellaSwag validation 23512","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to become an electrician in maine [title] Participate in a pre-apprenticeship in high school. [step] Maine offers a pre-apprenticeship program for students in 11th and 12th grade. The program requires students to log 1 , 000 hours of on-the-job electrician training by working part-time during the school year, and part-time or full-time during the summer.\",\"endings\":\"[\\\"Although the program is relatively easy, it can be challenging. To become an electrician in maine, you will want to take your ged or related courses to college level where you'll get an equivalent degree in electrical engineering.\\\", \\\"[title] Get certified as a electrician in maine. [step] The required courses can be found online using the skills skills calculator or the maine electrician association's website.\\\", 'Upon completion of the pre-apprenticeship program, graduates may enter an electrician apprenticeship program. Pre-apprenticeship hours count towards the hours required for the electrician apprenticeship in maine.', \\\"[substeps] Some operators require a bachelor's degree in carpentry or plumbing or an advanced degree in electrical engineering. These classes can also help you hone your electrical skills.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}