{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"arc-challenge","formal_name":"ARC-Challenge","introduction":"ARC-Challenge is the subset of grade-school science questions that both a retrieval baseline and a word co-occurrence baseline answered incorrectly. It tests reasoning with knowledge rather than recall alone.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/allenai/ai2_arc","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"da578015-9412-5e09-ac8a-7a2b2b912fc3","task_key":"ARC~2dChallenge--test--WASL~5f2005~5f8~5f12","task_revision_id":"1","upstream_id":"WASL_2005_8_12","short_description":"In your garden, you want to plant marigolds all around your pepper plants…","config":"ARC-Challenge","split":"test","body":"{\"choices\":\"['Count the number of whiteflies on the peppers before and after planting marigolds.', 'Count the number of whiteflies on the marigolds near the peppers.', 'Kill the whiteflies on other plants in the garden except on the peppers.', 'Put whiteflies on other garden plants and see what happens.']\",\"question\":\"In your garden, you want to plant marigolds all around your pepper plants because you read that this flower repels whiteflies. How can you test to see if marigolds really do repel whiteflies?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/allenai/ai2_arc","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}