{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"arc-challenge","formal_name":"ARC-Challenge","introduction":"ARC-Challenge is the subset of grade-school science questions that both a retrieval baseline and a word co-occurrence baseline answered incorrectly. It tests reasoning with knowledge rather than recall alone.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/allenai/ai2_arc","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"d7f5e5fd-3f08-5a55-9f39-7597eeea82cd","task_key":"ARC~2dChallenge--test--MCAS~5f2014~5f8~5f20","task_revision_id":"2","upstream_id":"MCAS_2014_8_20","short_description":"The students in a class would like to make 20 paper sailboats for a race. The…","config":"ARC-Challenge","split":"test","body":"{\"choices\":\"['construct a prototype of a boat for testing', 'calculate the total mass of all of the finished boats', 'determine the total amount of weight each boat can carry', 'test the strength of each material used to construct the boats']\",\"question\":\"The students in a class would like to make 20 paper sailboats for a race. The students will select one design and collect the materials they need to construct the boats. Which of the following is the best way for the students to be sure the paper sailboats will float without tipping over in the water?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/allenai/ai2_arc","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}