{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6fd7a59a-c740-504d-acba-99c2a6b4bfe1","task_key":"default--validation--21459","task_revision_id":"1","upstream_id":"21459","short_description":"HellaSwag validation 21459","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to take action to amend the constitution [title] Understand the process. [step] The united states constitution can only be amended through narrowly defined actions of the u.s. congress. Learn more about this process, for a start.\",\"endings\":\"['You will need to set your constitution aside in a respectful manner and treat it with respect. [substeps] Remember the implementation of a constitution from the creation of your constitution or a new policy.', '[substeps] Create an effective constitution that will strengthen the constitution and bring them into conflict resolution by improving the constitution. Take note that your supporting legal arguments will appear in your constitution, as well as your technical sources, legislation, and other information.', 'While you can take action to pressure congress into acting, congress itself must take the official steps. [substeps] Article 5 of the constitution governs how amendments are made.', '[substeps] The constitution can be amended based on a process of the constitution. Basically, your constitution goes into effect when the majority states that the constitution should be changed in 20 years.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}