{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e938e901-1a7e-536f-bb2b-edb62dc75a38","task_key":"default--validation--38541","task_revision_id":"1","upstream_id":"38541","short_description":"HellaSwag validation 38541","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to start college if you're blind or visually impaired [title] Attend orientation. [step] Before classes begin, colleges typically offer orientation sessions to help you get connected and acclimated to campus. During this time, you will likely be able to meet staff from the disability services office who will assist you greatly during your time in college.\",\"endings\":\"['You may also meet new friends who can help you out during this transitional. [substeps] This is also a great time to connect with faculty and staff.', \\\"[substeps] Local professors will be able to show you around campus and get you familiar with campus life. You'll also be able to connect with community organizations you were never aware of.\\\", \\\"[substeps] Many also offer counseling at regular intervals. If you ever feel you'd like assistance, discuss the issues with a counselor, or even with a guidance counselor.\\\", '[substeps] Orientation sessions can be like a tour of campus and accommodations. However, this does not mean you will have to attend yourself.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}