{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"arc-challenge","formal_name":"ARC-Challenge","introduction":"ARC-Challenge is the subset of grade-school science questions that both a retrieval baseline and a word co-occurrence baseline answered incorrectly. It tests reasoning with knowledge rather than recall alone.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/allenai/ai2_arc","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"3fdd4856-bc88-5fcb-9c86-59de094bbe85","task_key":"ARC~2dChallenge--test--Mercury~5f7247013","task_revision_id":"2","upstream_id":"Mercury_7247013","short_description":"Which of these serve as protective barriers along coastlines and are often…","config":"ARC-Challenge","split":"test","body":"{\"choices\":\"['the erosion of land along the coastline', 'the buildup of dunes along the coast over time', 'the new construction of buildings along the coastline', 'the buildup of organic material from oil rigs along the coast']\",\"question\":\"Which of these serve as protective barriers along coastlines and are often destroyed when hit by a strong hurricane?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/allenai/ai2_arc","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}