{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"arc-challenge","formal_name":"ARC-Challenge","introduction":"ARC-Challenge is the subset of grade-school science questions that both a retrieval baseline and a word co-occurrence baseline answered incorrectly. It tests reasoning with knowledge rather than recall alone.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/allenai/ai2_arc","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"0ac48a75-cad9-5c27-8b98-c1b1bf219688","task_key":"ARC~2dChallenge--test--Mercury~5f7168805","task_revision_id":"2","upstream_id":"Mercury_7168805","short_description":"Farmers in Wyoming were concerned because some of their chickens were being…","config":"ARC-Challenge","split":"test","body":"{\"choices\":\"['The chicken population would go down.', 'Populations of mice and rats would increase.', 'Another bird of prey would replace the hawk.', 'The chickens would have a lower rate of disease.']\",\"question\":\"Farmers in Wyoming were concerned because some of their chickens were being preyed upon by hawks that lived in areas around their ranches. The farmers grouped together and hunted the hawks until they were no longer in their area. Which would most likely happen next?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/allenai/ai2_arc","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}