{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0d110d49-1311-5cb4-a782-541c226439da","task_key":"default--validation--28724--variant-e274aa808d4c","task_revision_id":"1","upstream_id":"28724--variant-e274aa808d4c","short_description":"HellaSwag validation 28724","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to find the publisher of a book [title] Look for the publisher's name on the title page. [step] The name of the publisher should appear at the bottom of the title page, under the title and author of the book. The publisher's name will usually be listed under the logo for the publisher.\",\"endings\":\"['[substeps] The title of the book may appear under a suffix such as \\\" isbn \\\" or \\\" publisher \\\", or is preceded by a comma. It is usually listed below a comma for publisher.', \\\"[title] Check the copyright page for the publisher's name. [step] The copyright page may appear on the back of the title page or on the following page.\\\", 'The word \\\" publisher \\\" should also be in the name of the book. [substeps] The publisher should also be listed before the authors\\\\' names, at the beginning of each publication page.', \\\"Try to find the author's last name when you add the publisher. [substeps] If you are unsure, check the publisher's website for more information.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}