Discussion
0 posts
No posts in this thread yet.
benchmarks.wiki / Public workspace
SciCode
Science / Code
SciCode evaluates the ability to solve scientific research problems through code. Problems are decomposed into subproblems; this dev import preserves the relationships between 15 parent problems and 50 subproblems.
scicode
Full text: eligible public problems include text, images and discussion.
How it was built
SciCode structures scientific coding problems as parent tasks with linked subproblems. Records provide descriptions, scientific background, input and output information, dependencies and, when supplied, code templates. The catalogue development selection contains 15 parent problems and 50 subproblems, giving 65 task records rather than 65 independent research problems. Reference solutions and test fields are kept outside the displayed prompts.
At a glance
What a problem looks like
Parent scientific problems contain descriptions, background, input/output information, and dependencies. sub_steps become linked child tasks with step descriptions, background, input/output information, dependencies, and code templates when supplied.
How it is scored
For the 65 imported problems, the source ships an answer with every problem; this catalogue keeps it in restricted storage and never renders it.
Why it is hard
Requires solving scientific research problems through code, with parent problems decomposed into subproblems.
Limitations and criticism
Solving a supplied scientific coding task does not establish the ability to choose a research question or validate a scientific model against experimental evidence. Performance on an individual subproblem also does not show that a system can combine all the steps into a working parent solution. Results on this development selection should be distinguished from held-out evaluation, and parent and child outcomes should not be treated as independent observations.
Reference links
Discussion
1 thread and 0 posts on this page. Anyone may reply; no account is required.
Contribute
Share a reproduction, report an evaluation issue, or suggest a correction to this benchmark overview.
Tasks
Whole public benchmark.
- Public problems
- 65
- Formats
- scicode (15) · scicode-step (50)
- Problems with images
- 0
- Answer availability
- Answer published by the source (65)
- Configs
- Default (65)
- Splits
- dev (65)
- Awaiting a first discussion
- 65
- Problems with a discussion
- 0
- Public contributions
- 0
Provenance and terms
Pinned source revisions, newest first:
4510f6a6aa27c43fad7b43da2c59602a86e88480· acquired 2026-09-09 · release1993c24866354510f6a6aa27c43fad7b43da2c59602a86e88480· acquired 2026-09-09 · release25290d9a506e4510f6a6aa27c43fad7b43da2c59602a86e88480· acquired 2026-09-09 · release502aa46ac3e34510f6a6aa27c43fad7b43da2c59602a86e88480· acquired 2026-09-09 · release8507fdfa66614510f6a6aa27c43fad7b43da2c59602a86e88480· acquired 2026-09-08 · releasefb2b73197526
- CG
Answer published by the source
- ewald_summation
Answer published by the source
- The ligation of 2-mer ij depends on the concentration of 2-mer j'i', which is…
Answer published by the source
- Create a function to solve the matrix equation Ax=b using the Gauss-Seidel…
Answer published by the source
- Gram_Schmidt_orthogonalization
Answer published by the source
- Based on lennard jones potential with potential well depth epislon that reaches…
Answer published by the source
- Write a function to aggregate the net forces on each atom for a system of atoms…
Answer published by the source
- This function runs Velocity Verlet algorithm to integrate the positions and…
Answer published by the source
- Write a function that runs MD simulation for a system of atoms interacting…
Answer published by the source
- Gauss_Seidel
Answer published by the source
- Reciprocal_lattice_vector
Answer published by the source
- Using Newton's law …
Answer published by the source
- Write a function that that computes the right hand side of the evolution…
Answer published by the source
- Write a function that implements the 4th order accurate classical Runge-Kutta…
Answer published by the source
- IncomChol
Answer published by the source
- two_mer_entropy
Answer published by the source
- Internal_Energy
Answer published by the source
- Create a function to solve the linear system …
Answer published by the source
- nbody
Answer published by the source
- Write a function to compute the PMNS matrix of the three neutrino oscillation…
Answer published by the source
- Write a function to compute the expansion coefficients h_(k) of the three…
Answer published by the source
- Compute the expansion coefficients u_(k) of the evolution operator of the three…
Answer published by the source
- Write a function to compute the 3×3 evolution operator as defined in…
Answer published by the source
- SciCode_Example
Answer published by the source
- Write a Python function to determine the alpha value for the Ewald summation…
Answer published by the source
- Write a Python function to calculate the summation of the charge terms
Answer published by the source
- Write a Python function that calculates the energy of a periodic system using…
Answer published by the source
- Write a Python function to generate the tiled lattice coordinates from the…
Answer published by the source
- Write a Python function to evaluate the real-space summation as follows, where…
Answer published by the source
- Write a Python function to generate all possible reciprocal-space points that…
Answer published by the source
- Write a Python function to calculate the following weight at each reciprocal…
Answer published by the source
- Write a Python function to calculate this reciprocal-lattice sum, where the…
Answer published by the source
- Spatial_filters_II
Answer published by the source
- neutrino_oscillation
Answer published by the source
- Chaotic_Dynamics_Pendulum
Answer published by the source
- Create a function to compute the incomplete Cholesky factorization of an input…
Answer published by the source
- Write a fourth-order fixed-time-step Runge-Kutta (RK4) integrator from scratch…
Answer published by the source
- Now write a function to analyze any damped, driven pendulum system to understand…
Answer published by the source
- Write a function to get the total energy of a single atom, given the function…
Answer published by the source
- Write a function to get the total energy of the whole system, given the function…
Answer published by the source
- Write a function to use Markov Chain Monte Carlo simulation to generate samples…
Answer published by the source
- Provide a fucntion that normalizes the input vector. The input should be a numpy…
Answer published by the source
- Provide a function that computes the inner product of the two vectors in the…
Answer published by the source
- With the previous functions, provide a function that performs Gram-Schmidt…
Answer published by the source
- Given three vectors, provide a function that returns the reciprocal vectors of…
Answer published by the source
- Given some input vectors (could be one, two or three in total), provide the…
Answer published by the source
- Spatial filters are designed for use with lasers to "clean up" the beam.…
Answer published by the source
- Spatial filters are designed for use with lasers to "clean up" the beam.…
Answer published by the source
- (DUPLICATE) Write a function that returns the tensor product of an arbitrary…
Answer published by the source
- Compute the n-tangle of a n-qubit pure state |ψ⟩ where n is…
Answer published by the source