{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"arc-challenge","formal_name":"ARC-Challenge","introduction":"ARC-Challenge is the subset of grade-school science questions that both a retrieval baseline and a word co-occurrence baseline answered incorrectly. It tests reasoning with knowledge rather than recall alone.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/allenai/ai2_arc","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0fe650a3-ffe5-5cbf-96bc-81b1f744b61c","task_key":"ARC~2dChallenge--test--MCAS~5f2004~5f5~5f33","task_revision_id":"1","upstream_id":"MCAS_2004_5_33","short_description":"Few flowers are able to grow on the northern arctic tundra. Those that do grow…","config":"ARC-Challenge","split":"test","body":"{\"choices\":\"['It protects them from freezing.', 'It prevents them from being eaten by consumers.', 'It protects them from breaking in strong winds.', 'It makes it very hard for them to be pulled from the ground.']\",\"question\":\"Few flowers are able to grow on the northern arctic tundra. Those that do grow there have very short stems. How is this an adaptation to help them survive in the arctic climate?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/allenai/ai2_arc","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}