{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"47df744b-fcdd-59f8-aaf2-667975a2cf9f","task_key":"default--validation--40207--variant-40ac3a73ed8b","task_revision_id":"1","upstream_id":"40207--variant-40ac3a73ed8b","short_description":"HellaSwag validation 40207","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to find sources of international law [title] Look to treaties first. [step] Treaties are the main source of international law. Treaties can go by various names, such as international agreements, conventions, protocols etc.\",\"endings\":\"['Often, treaties declare that they will inform all parties. This agreement often is not too simple, especially when you are trying to identify the topic of the law.', 'But they are in essence the same thing-a written international agreement concluded between two or more countries. [title] Be careful to ensure that your country has signed and ratified the treaty in question before applying it to your domestic situation.', 'A treaties is a document which specifies the meaning and purpose of an object and the basic identification of it. When a treatise is compiled by the multiple nations, it should be grouped together and converted into separate pages.', 'That are related to the act or industry that led to the treaties. Always read information and see how a treaties explains the law.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}