{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"arc-challenge","formal_name":"ARC-Challenge","introduction":"ARC-Challenge is the subset of grade-school science questions that both a retrieval baseline and a word co-occurrence baseline answered incorrectly. It tests reasoning with knowledge rather than recall alone.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/allenai/ai2_arc","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"cb3c6bbd-4c90-5d85-bfcb-c032566f60b5","task_key":"ARC~2dChallenge--test--Mercury~5f7127943","task_revision_id":"2","upstream_id":"Mercury_7127943","short_description":"As early as the 1600s, scientists noticed that the continents appeared to fit…","config":"ARC-Challenge","split":"test","body":"{\"choices\":\"['to explain the mechanism that caused the continents to move', 'to consider the glacial evidence as outdated', 'to explain the extinction of the dinosaurs', 'to consider it unlikely that the continents were once together']\",\"question\":\"As early as the 1600s, scientists noticed that the continents appeared to fit together like pieces of a puzzle. The idea that the continents were once together and had drifted apart was not widely accepted. In the 1940s, technology used on submarines led to evidence of continental movement. This technology allowed the scientists to do which of these?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/allenai/ai2_arc","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}