benchmarks.wiki / Public workspace
ARC-Challenge
Reasoning / Science
ARC-Challenge is the subset of grade-school science questions that both a retrieval baseline and a word co-occurrence baseline answered incorrectly. It tests reasoning with knowledge rather than recall alone.
arc-challenge
Full text: eligible public problems include text, images and discussion.
Reference links
At a glance
What a problem looks like
A science question with three to five options. The options are imported as a list; the answer key is not.
How it is scored
The source ships an answer key with every problem; this catalogue keeps it in restricted storage and never renders it.
Why it is hard
The set was built by discarding everything the simple baselines could already do, so what remains needs a chain of reasoning over the science involved rather than a lookup.
Cite
@article{clark2018arcchallenge,
title={Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge},
author={Peter Clark and Isaac Cowhey and Oren Etzioni and Tushar Khot and Ashish Sabharwal and Carissa Schoenick and Oyvind Tafjord},
journal={arXiv preprint arXiv:1803.05457},
year={2018}
}Discussion
Contribute
Share a reproduction, report an evaluation issue, or suggest a correction to this benchmark overview.
Tasks
Whole public benchmark.
- Public problems
- 200
- Formats
- text (200)
- Problems with images
- 0
- Answer availability
- Answer published by the source (200)
- Configs
- ARC-Challenge (200)
- Splits
- test (200)
- Awaiting a first discussion
- 200
- Problems with a discussion
- 0
- Public contributions
- 0
Provenance and terms
Pinned source revisions, newest first:
210d026faf9955653af8916fad021475a3f00453· acquired 2026-09-12 · release64068a43aa57210d026faf9955653af8916fad021475a3f00453· acquired 2026-09-12 · releasea14a6faab45b
- Which best describes how the sun might compare to a different-sized star, if…
Answer published by the source
- Which environmental risk is not associated with the disposal of used motor oil…
Answer published by the source
- Which example shows a relationship between a living thing and a nonliving thing?
Answer published by the source
- Air has no color and cannot be seen, yet it takes up space. What could be done…
Answer published by the source
- Lasers are used in surgery for
Answer published by the source
- When the temperature of a sample of 25 water is -5°C, the water is
Answer published by the source
- The period of rotation for each planet in our solar system is different. The…
Answer published by the source
- How are sedimentary rocks made?
Answer published by the source
- A student mixes vinegar and baking soda to use in a volcano model. The mixture…
Answer published by the source
- What is the mass of a carbon atom that has 6 protons, 7 neutrons, and 6…
Answer published by the source
- Cars were once assembled completely by hand. Industrial robots now assemble many…
Answer published by the source
- The gravitational pull between the Moon and Earth affects conditions here on…
Answer published by the source
- Nutrient-laden waters from heavy rains flow into the Gulf of Mexico. This…
Answer published by the source
- Garden plants on Earth require four resources to stay alive: soil, air, water,…
Answer published by the source
- Hurricanes form over equatorial areas. This is because
Answer published by the source
- Why does a floor covered with tile feel colder than a floor covered with carpet?
Answer published by the source
- A doctor treats a patient showing symptoms of fatigue and fever. An examination…
Answer published by the source
- Petrified palm trees are found in sedimentary rock near glaciers. The presence…
Answer published by the source
- Which question about tulips could best be answered by scientific research?
Answer published by the source
- Students conduct an investigation with 20 of the same tomato plants. Each plant…
Answer published by the source
- If a nerve cell stopped functioning, the cell would
Answer published by the source
- Which process in the carbon cycle takes the longest to complete?
Answer published by the source
- Which part of an atom has the least mass?
Answer published by the source
- A lake that received runoff from surrounding farmland had a healthy fish…
Answer published by the source
- Farmers in Wyoming were concerned because some of their chickens were being…
Answer published by the source
- Students are observing two fossils in a lab. They are told by the teacher that…
Answer published by the source
- An astronomer observes that a planet rotates faster after a meteorite impact.…
Answer published by the source
- Experiments show that mice can learn to press a lever in order to acquire food.…
Answer published by the source
- The best way to separate salt from water is with the use of
Answer published by the source
- A bar magnet was moved quickly back and forth in a coil of copper wire. The ends…
Answer published by the source
- Which substance, when mixed with water, will produce the best conductor of…
Answer published by the source
- Which is most useful to a student who is separating aluminum screws from steel…
Answer published by the source
- Which action explains how a somatic cell mutation can harm an organism?
Answer published by the source
- What is the complementary base to cytosine in DNA?
Answer published by the source
- The morning temperature in a city is 41°F. If a sunny, mild day is forecast,…
Answer published by the source
- Some students are performing hardness tests on several substances. X scratches…
Answer published by the source
- Which is an example of a learned behavior?
Answer published by the source
- A carbon monoxide detector is used in homes to
Answer published by the source
- Some marine organisms living deep in the ocean can produce their own light. The…
Answer published by the source
- Soil texture is improved by
Answer published by the source
- Snow, rain, hail, and fog are all forms of
Answer published by the source
- Which statement best describes a property of water?
Answer published by the source
- A group of engineers wanted to know how different building designs would respond…
Answer published by the source
- A student is going to walk outside to observe her shadow at four different times…
Answer published by the source
- Why do most of the leaves of forest trees grow at the top of the tree?
Answer published by the source
- A polished metal ball looks very shiny and bright on a sunny day. What makes the…
Answer published by the source
- Which statement best describes the energy changes that occur while a child is…
Answer published by the source
- In the spring and early summer, bears often scratch their backs against trees to…
Answer published by the source
- Which of these meals would give you most of the nutrients that you need?
Answer published by the source
- If you throw each one of these things away, which will decay fastest?
Answer published by the source