{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"arc-challenge","formal_name":"ARC-Challenge","introduction":"ARC-Challenge is the subset of grade-school science questions that both a retrieval baseline and a word co-occurrence baseline answered incorrectly. It tests reasoning with knowledge rather than recall alone.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/allenai/ai2_arc","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"60797f17-c4b6-5dd9-8da1-13849a75a86a","task_key":"ARC~2dChallenge--test--Mercury~5f7128555","task_revision_id":"1","upstream_id":"Mercury_7128555","short_description":"In a certain field, plants with blue flowers are pollinated by bees, while…","config":"ARC-Challenge","split":"test","body":"{\"choices\":\"['The population of blue flowers will increase over time.', 'The population of white flowers will increase over time.', 'The moths will start pollinating the blue flowers.', 'The bees will start pollinating the white flowers.']\",\"question\":\"In a certain field, plants with blue flowers are pollinated by bees, while plants with white flowers are pollinated by moths. If the population of bees decreases, which will most likely occur?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/allenai/ai2_arc","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}