{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"arc-challenge","formal_name":"ARC-Challenge","introduction":"ARC-Challenge is the subset of grade-school science questions that both a retrieval baseline and a word co-occurrence baseline answered incorrectly. It tests reasoning with knowledge rather than recall alone.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/allenai/ai2_arc","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a49ee5ff-1f12-56b0-81ed-10fdbeb33d4d","task_key":"ARC~2dChallenge--test--Mercury~5fSC~5f401295","task_revision_id":"1","upstream_id":"Mercury_SC_401295","short_description":"Many desert animals are only active at night. How does being active only at…","config":"ARC-Challenge","split":"test","body":"{\"choices\":\"['They can see insects that light up at night.', 'Their bodies lose less water in the cool night air.', 'They are able to find more plant food by moonlight.', 'Their bodies absorb sunlight in the daytime while they sleep.']\",\"question\":\"Many desert animals are only active at night. How does being active only at night most help them survive in a hot desert climate?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/allenai/ai2_arc","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}