{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"arc-challenge","formal_name":"ARC-Challenge","introduction":"ARC-Challenge is the subset of grade-school science questions that both a retrieval baseline and a word co-occurrence baseline answered incorrectly. It tests reasoning with knowledge rather than recall alone.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/allenai/ai2_arc","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b8c61825-d65e-55cd-9369-eeb01f34583e","task_key":"ARC~2dChallenge--test--Mercury~5f7131530","task_revision_id":"1","upstream_id":"Mercury_7131530","short_description":"A student mixed some yellow sulfur powder with some iron filings. She was able…","config":"ARC-Challenge","split":"test","body":"{\"choices\":\"['a physical change because the iron reacted with the sulfur', 'a chemical change because a new substance was formed', 'a physical change because a compound was formed', 'a chemical change because a magnet had to be used']\",\"question\":\"A student mixed some yellow sulfur powder with some iron filings. She was able to take the iron out of the sulfur by using a magnet. She then remixed the iron and sulfur in a test tube and heated it. After it cooled, she removed the substance from the test tube but could not separate the iron from the sulfur using the magnet. Which type of change occurred in the material when it was heated?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/allenai/ai2_arc","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}