{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"arc-challenge","formal_name":"ARC-Challenge","introduction":"ARC-Challenge is the subset of grade-school science questions that both a retrieval baseline and a word co-occurrence baseline answered incorrectly. It tests reasoning with knowledge rather than recall alone.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/allenai/ai2_arc","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"db6353c4-b39f-557e-8f91-fdf6d131da9c","task_key":"ARC~2dChallenge--test--MCAS~5f2016~5f8~5f15","task_revision_id":"1","upstream_id":"MCAS_2016_8_15","short_description":"A college campus has so few parking spaces that cars are often lined up waiting…","config":"ARC-Challenge","split":"test","body":"{\"choices\":\"['The college could build an additional parking lot, gather parking data, and then receive input from community meetings.', 'The college could limit access to parking lots during busy hours, hire security officers to help restrict parking, and then gather parking data.', 'The college could gather parking data, receive input from community meetings, build an additional parking lot, and then gather more parking data.', 'The college could hire security officers to help restrict parking, gather parking data, receive input from community meetings, and then gather more data.']\",\"question\":\"A college campus has so few parking spaces that cars are often lined up waiting to park during class hours. Which of the following describes how the college could best solve the need for more parking spaces using the universal systems model?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/allenai/ai2_arc","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}