{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"18ac885c-4818-5760-a2bc-aeed32d20b52","task_key":"default--validation--22220","task_revision_id":"1","upstream_id":"22220","short_description":"HellaSwag validation 22220","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to design an online course [title] Research how others have taught similar courses. [step] There is nothing wrong with borrowing or remixing material that other teachers have used for similar courses. You can save yourself a lot of time and maximize the quality of your content by reading other curriculums and adapting it to your class.\",\"endings\":\"[\\\"[substeps] While you shouldn't feel as if you aren't learning in five minutes, you should evaluate the two courses that you are currently in. They are likely to overshadow one another and are likely to provide different advantages.\\\", 'Consider going to a teacher who has taught or who has brought you good feedback, but has taken the time to help you develop meaningful practice in your subject. [substeps] Who do you really know? That means that you will be evaluating the similarities between materials and methods of learning.', \\\"Use your preferred search engine to find available course outlines or ask other teachers you know that have taught the course. [substeps] Make sure that material you find online isn't licensed or you'll need to get permission to use it.\\\", \\\"It's likely that your peers or classmates have accepted standardized materials out of fear of taking risks or judging you because you weren't perfecting your material. However, you shouldn't rely upon peer-reviewed material alone to outline whether or not you're prepared to do so.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}