{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"arc-challenge","formal_name":"ARC-Challenge","introduction":"ARC-Challenge is the subset of grade-school science questions that both a retrieval baseline and a word co-occurrence baseline answered incorrectly. It tests reasoning with knowledge rather than recall alone.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/allenai/ai2_arc","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f9291bad-67c6-58b2-aace-d1fc21aac378","task_key":"ARC~2dChallenge--test--AKDE~26ED~5f2008~5f8~5f7","task_revision_id":"1","upstream_id":"AKDE&ED_2008_8_7","short_description":"Which method should be used to determine the type of popcorn that produces the…","config":"ARC-Challenge","split":"test","body":"{\"choices\":\"['Research articles about different ways to heat popcorn to make it pop.', 'Design an experiment that tests multiple types of popcorn using repeated trials.', 'Heat kernels of some types on an electric stove and heat other types on a gas stove.', 'Construct a mathematical equation that compares size and mass of kernels from each type.']\",\"question\":\"Which method should be used to determine the type of popcorn that produces the greatest number of popped kernels?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/allenai/ai2_arc","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}