{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4c1fad87-1a35-5bfe-ad68-db38537b151f","task_key":"default--validation--19421","task_revision_id":"1","upstream_id":"19421","short_description":"HellaSwag validation 19421","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to read a book for school and enjoy it [title] Figure out what type of book you like. [step] Think back to the books you have enjoyed. Make a list of the books you've loved.\",\"endings\":\"['What genre, and author was that book from? Was it written in the first, second or third person? Focus on what made you enjoy these book. [title] Before picking out your book, look back onto the list of books you may have made earlier.', \\\"When new or unsure what type of book to read, you can start with the basics and work your way up. Make note of any myths or historical figures that you'd prefer to read.\\\", \\\"Are there any books in your genre, so you can swap them for books to learn a bit more about your particular area? You can choose what category you want to focus on for your book. [substeps] Companies like sam's bazaar and charlie brown have books in almost every genre, including fiction, nonfiction, horror, and nonfiction.\\\", 'Try to identify which book is already on your shelf. This can help you get organized and to get into the mindset that what is on your shelf is not.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}