{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"arc-challenge","formal_name":"ARC-Challenge","introduction":"ARC-Challenge is the subset of grade-school science questions that both a retrieval baseline and a word co-occurrence baseline answered incorrectly. It tests reasoning with knowledge rather than recall alone.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/allenai/ai2_arc","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0306a698-8a32-5127-8c59-009dd8e9042a","task_key":"ARC~2dChallenge--test--Mercury~5f408922","task_revision_id":"1","upstream_id":"Mercury_408922","short_description":"A chemical called DDT was once used to kill insect pests. When investigations…","config":"ARC-Challenge","split":"test","body":"{\"choices\":\"['Different predictions about the chemical were made.', 'Tests performed on the chemical had different results.', 'Repeated tests on the chemical reached the same conclusion.', 'Different procedures were used to test the properties of the chemical.']\",\"question\":\"A chemical called DDT was once used to kill insect pests. When investigations showed this chemical was harmful to some types of birds, the use of DDT stopped. How was the scientific process best able to help scientists understand that DDT was harmful to birds?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/allenai/ai2_arc","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}