{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6eda6278-30c0-5afe-8abb-c8410244f486","task_key":"default--validation--44515","task_revision_id":"1","upstream_id":"44515","short_description":"HellaSwag validation 44515","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to prepare for a criminal trial [title] Hire an attorney. [step] After you have been charged, you will be arraigned. You can hire an attorney or have one appointed for you.\",\"endings\":\"['A background check is required to determine whether an attorney is an ideal candidate. [substeps] Generally, you should hire an attorney with expertise in criminal law.', 'Once you have an attorney, you can discuss whether or not you want to talk with the police. If you do, your attorney should be present.', 'Attorneys are often the ones you hire to make the first decisions. They will work with the opposing party to get you a pretrial verdict and settle your case for you.', 'If you find that an attorney is not doing his job effectively, try hiring an attorney from an established practice. [substeps] The lawyer you hire should not be experienced in criminal cases.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}