{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ec67ed8b-27b3-5ee2-a2de-25cfe9f6ec87","task_key":"default--validation--41502","task_revision_id":"1","upstream_id":"41502","short_description":"HellaSwag validation 41502","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to teach reading [title] Teach the alphabet. [step] The first step in learning to read is recognizing the letters of the alphabet. Use a poster, chalkboard, or notebook to write or display the alphabet.\",\"endings\":\"['Go over the letters with the student until he or she learns them all. Use the alphabet song to help the student to remember.', '[substeps] Name all letters in the alphabet for use in the classroom. Place them in contrasting colors, rather than all at the same time.', 'Pay special attention to each letter in the alphabet as this will help you recognize everything you need to know. Some methods for learning the alphabet are : [substeps] Creative writing.', 'This will help you to put them in context. List the letters first, then move on to rhyming them out loud.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}