{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"arc-challenge","formal_name":"ARC-Challenge","introduction":"ARC-Challenge is the subset of grade-school science questions that both a retrieval baseline and a word co-occurrence baseline answered incorrectly. It tests reasoning with knowledge rather than recall alone.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/allenai/ai2_arc","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8cab0761-f236-5bc6-aa7d-ac888723cb7a","task_key":"ARC~2dChallenge--test--Mercury~5f7174738","task_revision_id":"1","upstream_id":"Mercury_7174738","short_description":"Gulf sturgeons, a type of fish, live in saltwater along the gulf coast of…","config":"ARC-Challenge","split":"test","body":"{\"choices\":\"['Gulf sturgeons will change over time into a new species.', 'Gulf sturgeons will move from coastal rivers into the ocean.', 'The gulf sturgeon population will decline due to loss of habitat.', 'The gulf sturgeon population will adapt to breeding in saltwater.']\",\"question\":\"Gulf sturgeons, a type of fish, live in saltwater along the gulf coast of Florida. They swim upriver to breed in clean freshwater. After several years, the young sturgeons migrate downriver to the coastal saltwater environments. Which would be the most likely effect on gulf sturgeons if dams were built along these rivers?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/allenai/ai2_arc","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}