{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fc0131b4-e15c-58c1-9474-702a0568a000","task_key":"all--test--fc0131b4-e15c-58c1-9474-702a0568a000","task_revision_id":"1","upstream_id":"","short_description":"Bob Wilson borrowed $20,000 from Ted Lamar to open a hardware store. Ted's only…","config":"all","split":"test","body":"{\"choices\":\"['only Bob Wilson is liable.', 'Bob Wilson and Ted Lamar are liable jointly.', 'Bob Wilson is liable for the entire amount and Ted Lamar is liable only to the extent the debt cannot be collected from Bob Wilson.', 'only the de facto partnership arising from the relationship between Wilson and Lamar is liable.']\",\"question\":\"Bob Wilson borrowed $20,000 from Ted Lamar to open a hardware store. Ted's only interest in the business was the repayment of his 5-year unsecured loan. Bob was so grateful for the loan that he named his business \\\"Wilson and Lamar Hardware\\\" and purchased signs and advertising displaying this name. He also listed Bob Wilson and Ted Lamar as \\\"partners\\\" on his stationery. When Ted found out, he was flattered to the point that he voluntarily reduced Bob's interest rate from 9 percent to 8 percent per annum. A few weeks later, Pete Smith, who had assumed that both Wilson and Lamar were operating the hardware store and was not familiar with the true situation, sold goods to Wilson and Lamar Hardware. Pete Smith has been unable to collect for the goods and he seeks your advice. Your advice to Pete is\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}