{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8813b59d-1660-5e80-b763-a09a2a374da7","task_key":"default--validation--44844","task_revision_id":"1","upstream_id":"44844","short_description":"HellaSwag validation 44844","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to fall asleep in class [title] Wait until class has started. [step] If the bell hasn't rung and you are already dozing off, you're more likely to draw attention to yourself. Hold off until roll call is finished and a few minutes of discussion has passed.\",\"endings\":\"[\\\"[title] Consider sending a brief email to your teacher if you're confused. [step] Anything pertaining to the subject matter you'll be studying that day doesn't belong to the class.\\\", \\\"This tactic allows the teacher to feel confident that their students are paying attention. Once the first few minutes of class have passed, you'll have a better chance at sleeping undetected.\\\", \\\"An hour of talking during class can help you raise your blood oxygen levels, which can help you slide to the next stage of sleep/snoozing. [title] Know your class's bedtime, not ones you normally enjoy.\\\", 'The process of falling asleep requires focused concentration. Additionally, wait until at least an hour has passed to nap as long as possible before you drift off.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}