{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"arc-challenge","formal_name":"ARC-Challenge","introduction":"ARC-Challenge is the subset of grade-school science questions that both a retrieval baseline and a word co-occurrence baseline answered incorrectly. It tests reasoning with knowledge rather than recall alone.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/allenai/ai2_arc","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ee009ae6-a868-5206-a5b4-3914d16fbe00","task_key":"ARC~2dChallenge--test--Mercury~5f7130655","task_revision_id":"1","upstream_id":"Mercury_7130655","short_description":"Until recently, only a small part of the human DNA was known to have a specific…","config":"ARC-Challenge","split":"test","body":"{\"choices\":\"['It will make the study of DNA much simpler.', 'It proves that DNA is less important than originally thought.', 'It provides evidence that scientific results should not be challenged.', 'It demonstrates that scientific knowledge is subject to change.']\",\"question\":\"Until recently, only a small part of the human DNA was known to have a specific function. The rest of the DNA was thought to be unimportant. New evidence now suggests that all DNA is needed in certain gene functions. Which statement best describes the significance of this new evidence?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/allenai/ai2_arc","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}