{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2e9f8b7f-8276-5f14-95c5-6a6708580c34","task_key":"default--validation--18700","task_revision_id":"1","upstream_id":"18700","short_description":"HellaSwag validation 18700","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to be prepared for a pop quiz [title] Get enough sleep. [step] Sleeping is an essential element in the learning process. Make sure to get your recommended seven to eight hours of sleep per night.\",\"endings\":\"['[title] Write down every topic before the quiz day. [step] Use it as a work paper, or as part of a interview or demonstration.', \\\"In most tests (and reading) by 10 pm, adults can fall asleep much faster than less-basic questions. In nearly all situations, like quizzes, it's best to get a good night's sleep before the test.\\\", 'Some people cannot tolerate seven day sleep due to circadian rhythm disorders such as migraines or high blood pressure. [substeps] It can also affect other factors, such as eye pressure or heartburn.', 'Getting adequate sleep will help your brain process the information you learned the day before. Therefore, you will be able to remember the information better.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}