{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"83ffd039-9c57-580a-8eb5-7aab41e3cabe","task_key":"default--validation--15506","task_revision_id":"1","upstream_id":"15506","short_description":"HellaSwag validation 15506","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to advocate for your child with special needs [title] Take a local workshop or online course about special education law. [step] Local community colleges or universities often offer workshops about special education laws, programs, and resources. Enroll in one of these or take a class online.\",\"endings\":\"['[substeps] Some programs may offer certification while others are simply informative. Check around to find a program that suits your goals.', 'You can check the website of a national organization, such as the american academy of special education, to find out additional information about special education. [substeps] Many centers and colleges offer special education classes, so talk to your local community college or university to see if they have a workshop of special education law.', 'Study with those in your community during workshops to learn more about the law and different classes involved. [substeps] That being said, look for courses in the school or community center that focus on learning certain principles, procedures, and terms related to special education law.', 'You can often find community classes geared specifically to special education families. [substeps] Many community colleges and universities also offer courses in special education law.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}