{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"dbf73de1-20db-5137-9a77-173df9624e67","task_key":"default--validation--41644--variant-07b4781a9b61","task_revision_id":"1","upstream_id":"41644--variant-07b4781a9b61","short_description":"HellaSwag validation 41644","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to end a sporting facility contract [title] Find your sporting facility contract. [step] Your contract is your legal agreement which you must abide by. If you fail to uphold your end of the bargain, then the other party can sue you in court and get money damages.\",\"endings\":\"['Accordingly, you must follow your obligations in the lease agreement. [substeps] Before you can do anything, you should find your copy of the lease.', 'You should search for the written contract at. If you cannot find your agreement online, then you can perform an internet search.', 'Make sure your contract states that you uphold the terms that you signed when you started your sport. [substeps] Generally, your agreement is not binding, but you may deduct some money by violating it.', \\\"If you don't uphold your end of the bargain, you will be accused of a misconduct that could put you in prison, or may even be compensatory damages against you. The agreement might state : [substeps] How the other side can win.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}