{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9c41e354-fcd9-5987-bbe1-1fc07a9d5957","task_key":"all--test--9c41e354-fcd9-5987-bbe1-1fc07a9d5957","task_revision_id":"1","upstream_id":"","short_description":"A grain broker specializes in exotic grains for the growing organic-food market.…","config":"all","split":"test","body":"{\"choices\":\"['admissible as evidence of customary business practices in the community.', 'admissible as a proper lay opinion.', 'inadmissible as hearsay.', 'inadmissible as irrelevant.']\",\"question\":\"A grain broker specializes in exotic grains for the growing organic-food market. Using his network of contacts across the globe, he is able to find sources for almost any kind of grain, grown anywhere in the world, to be shipped anywhere in the world. Last year, an organic food co-op asked the grain broker to find quinoa for them. The grain broker said, \\\"I'll do it. It'll cost you market rate plus the commission. \\\" The grain broker found the quinoa and had it shipped to the co-op at a cost of $15 per bushel plus a 20% commission. The co-op paid $15 per bushel plus an 8% commission. The grain broker brought suit, claiming that the appropriate commission was 20%. The grain broker introduced business records and other documentary evidence establishing that he normally charges a commission of between 10% and 20% for grains, with a higher commission for more exotic grains. The owner of the co-op called another grain broker, a competitor of the plaintiff, to testify. The witness testified that he never charges more than an 8% commission when asked to locate exotic grains for his customers. The trial judge should rule this testimony\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}