{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"arc-challenge","formal_name":"ARC-Challenge","introduction":"ARC-Challenge is the subset of grade-school science questions that both a retrieval baseline and a word co-occurrence baseline answered incorrectly. It tests reasoning with knowledge rather than recall alone.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/allenai/ai2_arc","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b822cc48-19be-59bb-9a05-c48dd459f498","task_key":"ARC~2dChallenge--test--MCAS~5f2006~5f9~5f4","task_revision_id":"1","upstream_id":"MCAS_2006_9_4","short_description":"There are two types of modern whales: toothed whales and baleen whales. Baleen…","config":"ARC-Challenge","split":"test","body":"{\"choices\":\"['Primitive whales had teeth as adults.', 'Toothed whales descended from baleen whales.', 'Baleen whales are evolving into toothed whales.', 'Descendants of modern baleen whales will have both teeth and baleen as adults.']\",\"question\":\"There are two types of modern whales: toothed whales and baleen whales. Baleen whales filter plankton from the water using baleen, plates made of fibrous proteins that grow from the roof of their mouths. The embryos of baleen whales have teeth in their upper jaws. As the embryos develop, the teeth are replaced with baleen. Which of the following conclusions is best supported by this information?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/allenai/ai2_arc","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}