{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"98402beb-b2ac-5217-bda0-9b6b9676e64d","task_key":"default--validation--41980","task_revision_id":"1","upstream_id":"41980","short_description":"HellaSwag validation 41980","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to help prevent nuclear war [title] Show up to town hall meetings, if applicable. [step] If you have the opportunity to go to a town hall meeting in your city or town, use this as a chance to voice your concerns about the threat of nuclear war. The representatives at this meeting will be able to hear your message, as well as others attending the town hall meeting.\",\"endings\":\"[\\\"[substeps] Prepare what you're going to say before attending the meeting. Use the information you gathered in your research to support your argument.\\\", '[title] Discuss program issues if conditions or passions stop your ideas. [step] Discuss local issues that you are worried about and what is causing them.', \\\"[title] Help to make cleanup less disruptive to the surrounding environment. [step] You don't have to let the public know about the threat, of course.\\\", '[substeps] Make sure to introduce yourself and outline your cause. For example, say something like, \\\" i\\\\'m a citizen of america and i want to help in the developing world.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}