{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"43fcde3d-ebc7-5e4c-82e1-32fdfcad262a","task_key":"default--validation--6949","task_revision_id":"1","upstream_id":"6949","short_description":"HellaSwag validation 6949","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to write a residential lease [title] Check with an attorney. [step] If you are writing a new residential lease form from scratch, it may be helpful to work with an attorney. Some lease provisions will be required by the laws of your state, while some provisions that are required in one state may be illegal in another.\",\"endings\":\"[\\\"Working with an experienced real estate attorney will help you make sure that your lease will be enforceable in your state. [substeps] Typically, if one provision isn't legal, then that provision won't be enforced but the rest of the lease will be.\\\", '[substeps] To find an attorney, you can visit your state bar association. Call up and ask for a referral, or run your research.', \\\"[substeps] You should read and understand your state's laws carefully before making a request for a residential lease. Federal law prohibits qualified attorneys from drafting residential leases.\\\", \\\"Some states only require these provisions in certain areas. If you're unable to find a lawyer familiar with property law, discuss your options with your local or state bar association.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}