benchmarks.wiki / Public workspace
MMLU-Pro
Reasoning
MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.
mmlu-pro
Full text: eligible public problems include text, images and discussion.
Reference links
At a glance
What a problem looks like
Multiple-choice questions with ten options. The import accepts question, options, category and src.
How it is scored
For the 200 imported problems, the source ships an answer with every problem; this catalogue keeps it in restricted storage and never renders it.
Why it is hard
Ten options rather than four, and the easiest original items removed, so guessing and memorisation carry much less of the score.
Discussion
Contribute
Share a reproduction, report an evaluation issue, or suggest a correction to this benchmark overview.
Tasks
Whole public benchmark.
- Public problems
- 200
- Formats
- text (200)
- Problems with images
- 0
- Answer availability
- Answer published by the source (200)
- Configs
- default (200)
- Splits
- test (200)
- Awaiting a first discussion
- 200
- Problems with a discussion
- 0
- Public contributions
- 0
Provenance and terms
Pinned source revisions, newest first:
b189ec765aa7ed75c8acfea42df31fdae71f97be· acquired 2026-09-11 · releasef17f6fbd935b
- A man sells novelty items for 1.25each.Hiscostis.75 apiece plus a fixed…
Answer published by the source
- A man buys a house and lot for 35,000,paying12,000 down and borrowing the…
Answer published by the source
- Mr. Castle will buy one of two 10-HP motors offered to him. Motor A sells for…
Answer published by the source
- Which of the following would yield the greatest net return to acorporation in…
Answer published by the source
- Assume the following model (from the preceding problem). Y = C + I + G C = 100 +…
Answer published by the source
- The average salary of the Bedford Manufacturing Co. rose 60% during a 10-year…
Answer published by the source
- In a business to business context, conflicts of interest can arise in two main…
Answer published by the source
- This is a hierarchy of effects or sequential model used to explain how…
Answer published by the source
- For a two-period binomial model for stock prices, you are given: (i) Each period…
Answer published by the source
- lf 10% and 20% discounts are both given on the" Speedway Racing Set",…
Answer published by the source
- What percent of .65% is .42%?
Answer published by the source
- George put …
Answer published by the source
- Paul Reilly deposited a …
Answer published by the source
- With the invoice amount of …
Answer published by the source
- .01(1/4)ashareforstocksunder5 a share par value …
Answer published by the source
- TencerInc. has estimated its revenue function to be r(x) = 3x^2, where x is the…
Answer published by the source
- A draft for …
Answer published by the source
- In one year Juan earned 30,000.andDonearned20,000 as free-lance commercial…
Answer published by the source
- A 39.90radiocanbepurchasedontheinstallmentplanfor5.00 down and …
Answer published by the source
- John Wilson retired at age 62 with average yearly earnings of …
Answer published by the source
- A sum of money is borrowed for a term of 1 year, discounted at the rate of 6%.…
Answer published by the source
- Steven Fox is a salesman for the Maxi-Power Vacuum Cleaner Co. He receives…
Answer published by the source
- Which of the following is not one the underlying principles of the corporate…
Answer published by the source
- There are three main types of buying situations in an organization, referred to…
Answer published by the source
- With this pricing approach, the pricing process begins with the customer; not…
Answer published by the source
- ___________ is based on the idea that customer expectations of the service they…
Answer published by the source
- What is the term for organisational development through a formally designed…
Answer published by the source
- Universal Fur is located in Clyde, Baffin Island, and sells high-quality fur bow…
Answer published by the source
- A teacher wants to invest …
Answer published by the source
- Mr. Singer insured his house for …
Answer published by the source
- Distribution of overhead is based on floor space. Department Aoccupied 8,100 sq.…
Answer published by the source
- Mr. Atlas had a note for …
Answer published by the source
- Two pumping systems are being considered for use on a project. The gasoline pump…
Answer published by the source
- John's-Rent-A-Truck purchased a van for …
Answer published by the source
- On January 1, 1964, Stead and Company purchased a delivery truck for …
Answer published by the source
- The McDougall Company produces and sells two qualities of carpet. The better…
Answer published by the source
- (A) How many gallons are there in 12 liters? (B) How many liters are there in 10…
Answer published by the source
- _______, where actors are convinced that their actions harms nobody, _______…
Answer published by the source
- Who of the following is a leading writer on contingency theory of leadership?
Answer published by the source
- In what order do managers typically perform the managerial functions?
Answer published by the source
- Consider an arbitrage-free securities market model, in which the risk-free…
Answer published by the source
- There are two main issues associated with _____ sizing. _______ is a key issue…
Answer published by the source
- Some of key differences between Islamic finance and conventional finance include…
Answer published by the source
- Where the price is set low relative to the competition to gain market share,…
Answer published by the source
- Once a train pulls out of a station, or an aeroplane takes off or a film starts,…
Answer published by the source
- A marketing research firm contracts with clients to conduct a complete marketing…
Answer published by the source
- What is the term for the 'rule of thumb' type of bias in decision making?
Answer published by the source
- The stock of the CCC Corporation is currently valued at …
Answer published by the source
- On July 7, Magee Data stock sold at a high of 23(1/8) and a low of 22(5/8).…
Answer published by the source
- Prepare a balance sheet for Silvertown Office Supplies, C.H. Walters, owner, as…
Answer published by the source