{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"arc-challenge","formal_name":"ARC-Challenge","introduction":"ARC-Challenge is the subset of grade-school science questions that both a retrieval baseline and a word co-occurrence baseline answered incorrectly. It tests reasoning with knowledge rather than recall alone.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/allenai/ai2_arc","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"24cfe836-1507-5bfa-9e40-5cb3095f8102","task_key":"ARC~2dChallenge--test--Mercury~5f7094308","task_revision_id":"1","upstream_id":"Mercury_7094308","short_description":"Several species of fish live in a classroom aquarium. Students notice that the…","config":"ARC-Challenge","split":"test","body":"{\"choices\":\"['Fish have a two-chambered heart.', 'Some fish are much larger than others.', 'Fish need different amounts of oxygen.', 'There are six types of fish in the aquarium.']\",\"question\":\"Several species of fish live in a classroom aquarium. Students notice that the gills of one type of fish move faster than the gills of the other fish. Which of these statements is the best hypothesis on this observation?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/allenai/ai2_arc","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}