{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"arc-challenge","formal_name":"ARC-Challenge","introduction":"ARC-Challenge is the subset of grade-school science questions that both a retrieval baseline and a word co-occurrence baseline answered incorrectly. It tests reasoning with knowledge rather than recall alone.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/allenai/ai2_arc","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"aa4b2042-ddbe-5c90-9612-64ed87ba5dbe","task_key":"ARC~2dChallenge--test--Mercury~5f7207550","task_revision_id":"1","upstream_id":"Mercury_7207550","short_description":"For many years, scientists have found fossils in rock layers in all parts of the…","config":"ARC-Challenge","split":"test","body":"{\"choices\":\"['fossils are preserved in different ways.', 'conclusions about fossils are revised into questions.', 'fossils are formed in different areas.', 'new fossil evidence is reviewed.']\",\"question\":\"For many years, scientists have found fossils in rock layers in all parts of the world. Eventually scientists discovered that many of the rock layers showed the same pattern of older fossils being located in deeper rock layers and younger fossils being located in shallower layers. This discovery is an example of how scientific knowledge changes as\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/allenai/ai2_arc","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}