{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"arc-challenge","formal_name":"ARC-Challenge","introduction":"ARC-Challenge is the subset of grade-school science questions that both a retrieval baseline and a word co-occurrence baseline answered incorrectly. It tests reasoning with knowledge rather than recall alone.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/allenai/ai2_arc","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5792d9a7-e1b8-50e3-af34-4f534602186b","task_key":"ARC~2dChallenge--test--Mercury~5f417593","task_revision_id":"1","upstream_id":"Mercury_417593","short_description":"A fish in a pond has longer protective spines than any other fish of its…","config":"ARC-Challenge","split":"test","body":"{\"choices\":\"['A mutation caused a beneficial change in the fish.', 'A mutation caused a harmful change in the fish.', 'Longer spines caused a genetic change in the fish.', 'Longer spines caused a temporary change in the fish.']\",\"question\":\"A fish in a pond has longer protective spines than any other fish of its species. The spines keep the fish safe from predators. The fish has many offspring, which also have long spines. Which statement best describes what happened?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/allenai/ai2_arc","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}