{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"arc-challenge","formal_name":"ARC-Challenge","introduction":"ARC-Challenge is the subset of grade-school science questions that both a retrieval baseline and a word co-occurrence baseline answered incorrectly. It tests reasoning with knowledge rather than recall alone.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/allenai/ai2_arc","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"157572e1-2edc-5624-9f27-dc1a1ac264ee","task_key":"ARC~2dChallenge--test--Mercury~5f7188580","task_revision_id":"1","upstream_id":"Mercury_7188580","short_description":"Golden eagles can see small objects from great distances. Which behavior of…","config":"ARC-Challenge","split":"test","body":"{\"choices\":\"['Hunting occurs while flying through the air.', 'They build their nests on cliffs or rock outcroppings.', 'Offspring depend upon parent feeding after hatching.', 'The color of their feathers lacks environmental camouflage.']\",\"question\":\"Golden eagles can see small objects from great distances. Which behavior of golden eagles best explains why natural selection most likely favored this structural adaptation?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/allenai/ai2_arc","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}