{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"arc-challenge","formal_name":"ARC-Challenge","introduction":"ARC-Challenge is the subset of grade-school science questions that both a retrieval baseline and a word co-occurrence baseline answered incorrectly. It tests reasoning with knowledge rather than recall alone.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/allenai/ai2_arc","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"82885eb1-6de5-5181-96da-de277f0e8ee7","task_key":"ARC~2dChallenge--test--MCAS~5f2014~5f5~5f7","task_revision_id":"2","upstream_id":"MCAS_2014_5_7","short_description":"A type of small mammal from the mountain regions of the western United States…","config":"ARC-Challenge","split":"test","body":"{\"choices\":\"['to repare for migration before winter', 'to provide warmth during the cold winter months', 'to store food that will be eaten over the winter months', 'to protect the grasses and seeds from decay before winter']\",\"question\":\"A type of small mammal from the mountain regions of the western United States makes its home out of piles of rock. During summer months, the mammal places grasses and seeds in protected places in the rock piles. Which of the following is the most likely reason for this behavior?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/allenai/ai2_arc","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}