{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9f9f2f8e-c922-5ed6-b8a9-6b147f22a4e4","task_key":"default--validation--5674","task_revision_id":"1","upstream_id":"5674","short_description":"HellaSwag validation 5674","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to dispute a lot line survey as a defendant [title] Hire a lawyer. [step] As soon as you are served with the plaintiff's lawsuit, you should strongly consider hiring a qualified real estate lawyer. Specifically, look for a lawyer with experience litigating property boundary disputes.\",\"endings\":\"['[substeps] While you are at the site, ask if your city or county bar association has a lawyer you can use. Alternatively, visit the website of each city or county bar association.', 'If you absolutely must hire a lawyer, then find one that specializes in property, design, or engineering. A general rule of thumb is to hire an experienced real estate lawyer, preferably who can provide personal experience to expert witnesses.', '[substeps] Never hire an attorney who does not work for the state or federal government. Only hire lawyers who have experience representing local or state judges and arbitrators.', 'A lawyer will help you negotiate a settlement, effectively cross-examine plaintiff witnesses, and retain a quality expert of your own. Start the search process by asking friends and family for recommendations.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}