{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9cbad494-915f-59b2-990e-56c15bd4e929","task_key":"default--validation--27805","task_revision_id":"1","upstream_id":"27805","short_description":"HellaSwag validation 27805","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to write a book report [title] Follow the requirements of your assignment. [step] Read through the assignment sheet carefully and make note of any questions that you have. Raise your hand during class or talk with your teacher afterward to go over any concerns.\",\"endings\":\"[\\\"Make sure that you know the required paper length, due date, and any formatting requirements, like double-spacing. [substeps] For example, you'll need to find out if your teacher wants you to include citations, such as page numbers from the book, in your paper.\\\", 'A good rule of thumb is to do this as normal and as soon as you take a reading of your book, you should open a book or report. Only complete the assignment if you are strictly at the grammar and follow the classroom rules.', '[substeps] College courses sometimes involve english and constitutional essays as well as psychological papers. Reading over your report before turning it in could help you to identify gaps that you may have missed.', \\\"The material you'll need to read is vital to your report, so make sure to keep it ahead of other subjects. It's up to you whether it seems simple or complex, but keep track of any questions that you have to make a point of having answered earlier in the week.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}