{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7ac48096-d225-50bf-9ef0-cf2b6e469593","task_key":"default--validation--35423","task_revision_id":"1","upstream_id":"35423","short_description":"HellaSwag validation 35423","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to be a cute geeky girl [title] Read! [step] If you're a geeky girl, you're a smart girl. Books give you lots of knowledge, making you smarter every time you end that last page. Reading can actually be a fun and interesting activity, if you choose the right books.\",\"endings\":\"[\\\"And you won't feel like a total nerd while reading. You can do by spending time reading, surfing the internet, watching websites, and walking around a library.\\\", \\\"You don't have to read every word of every book on the internet. You can read every book you want on the computer, or just watch your favorite movie.\\\", \\\"Books keep you entertained while the ice cream is melting and you're reading your book. [title] When reading, pick the right genre.\\\", 'Try reading different genres of books, and finding your favourite one. Reading passes lots of time, so try to read as often as possible.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}