{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"arc-challenge","formal_name":"ARC-Challenge","introduction":"ARC-Challenge is the subset of grade-school science questions that both a retrieval baseline and a word co-occurrence baseline answered incorrectly. It tests reasoning with knowledge rather than recall alone.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/allenai/ai2_arc","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fb05fe6e-cae4-5cc6-affc-8ae2fb3c80b8","task_key":"ARC~2dChallenge--test--MCAS~5f2006~5f8~5f24","task_revision_id":"1","upstream_id":"MCAS_2006_8_24","short_description":"A glass is partially filled with water. Five ice cubes are placed in the glass,…","config":"ARC-Challenge","split":"test","body":"{\"choices\":\"['The volume of the submerged ice is equal to the volume of water displaced.', 'The mass of the water in the glass is less than the mass of the ice.', 'The weight of the ice is less than the weight of the water in the glass.', 'The density of the water in the glass is greater than the density of the ice.']\",\"question\":\"A glass is partially filled with water. Five ice cubes are placed in the glass, causing the level of the water to reach the rim of the glass. Which of the following statements best explains the increase in water level?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/allenai/ai2_arc","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}