{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"arc-challenge","formal_name":"ARC-Challenge","introduction":"ARC-Challenge is the subset of grade-school science questions that both a retrieval baseline and a word co-occurrence baseline answered incorrectly. It tests reasoning with knowledge rather than recall alone.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/allenai/ai2_arc","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"f92912b7-e2f9-5650-a47a-2dcc6b76a9bc","task_key":"ARC~2dChallenge--test--Mercury~5f7234308","task_revision_id":"2","upstream_id":"Mercury_7234308","short_description":"A scientist maps a long region in which earthquakes originate and determines…","config":"ARC-Challenge","split":"test","body":"{\"choices\":\"['Volcanism also characterizes the region.', 'Earthquake centers in the region occur at shallow depths.', 'The region shows extensive faulting of sediments.', 'Equal crust densities are found on opposite sides of the region.']\",\"question\":\"A scientist maps a long region in which earthquakes originate and determines this region is a transform plate boundary. Which evidence would cause the scientist to reevaluate this determination?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/allenai/ai2_arc","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}