{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a003edb5-5e4c-5937-8cf2-dff56a94c6a2","task_key":"default--validation--25291","task_revision_id":"1","upstream_id":"25291","short_description":"HellaSwag validation 25291","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to decide what to podcast about [title] Discuss the news and/or current events. [step] Why not podcast on your opinions and thoughts of what's happening around the world, or in your own district? [title] Showcase your hobbies and/or interests. [step] This is the most popular form of podcast.\",\"endings\":\"['The important thing to discuss is what (if) you like to do. If you enjoy running and/or cycling, be sure to mention that.', 'Sharing your activities and experiences in this way will be very fun and can win you people respect. Many podcasts specify this as sharing your knowledge on a topic.', 'Why not talk about your favourite television show or review your favourite films? Or podcast about your favourite sport, or maybe you have an interesting or unusual pastime. These are good because when you talk excitedly about your interests, your listeners will became excited too.', 'There are a wide variety of coming-of-age podcasts released by many of the lower performers. The only new talent, however, will be their competitive nature.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}