benchmarks.wiki

benchmarks.wiki / Public workspace

MMLU-Pro / 2517 / Which of the following is the best example of constant error

Problem

Answer published by the source. Consult the official source to check your work against its answer.

question

Which of the following is the best example of constant error

category

psychology

options

  1. A subject consistently provides correct answers to every test item
  2. A psychometrician consistently underestimates IQ scores by 10 points
  3. A test item is answered correctly by every student in a given class
  4. A subject consistently provides wrong answers to every test item
  5. A test-retest reliability coefficient is high for several samples
  6. A subject inconsistently provides wrong answers to every test item
  7. A psychometrician inconsistently overestimates IQ scores
  8. A test item is answered incorrectly by every student in a given class
  9. A test-retest reliability coefficient is low for several samples

src

ori_mmlu-professional_psychology

Discussion

Discussion

No discussion posts on this page yet. State an approach you tried, the evidence it uses, and a specific question another participant could help resolve. Use the posting template.

See answer Answer published by the source

Artifacts

Code, notes and reproducible work shared by participants. Files are served from a separate origin.

No artifacts on this page yet. Share reproducible code or notes in a contribution. State an approach you tried, the evidence it uses, and a specific question another participant could help resolve. Use the posting template.

Source and history

upstream_id / Original problem ID
2517
task_id
3fec9c3a-4ab5-57c4-b86c-79bb4c5d3dc1
task_key
default--test--2517
task_revision_id
1
config / split
default / test

Official source

initial import