{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"arc-challenge","formal_name":"ARC-Challenge","introduction":"ARC-Challenge is the subset of grade-school science questions that both a retrieval baseline and a word co-occurrence baseline answered incorrectly. It tests reasoning with knowledge rather than recall alone.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/allenai/ai2_arc","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"dfee0547-6d62-5333-a1ab-f5439d4d0570","task_key":"ARC~2dChallenge--test--Mercury~5f7030468","task_revision_id":"1","upstream_id":"Mercury_7030468","short_description":"New technologies have allowed automobile manufacturers to market fuel-efficient…","config":"ARC-Challenge","split":"test","body":"{\"choices\":\"['What will be done to recycle the batteries?', 'What will be done with the refrigerant in the air conditioning system?', 'What will be done with the fluids in the gas tank, the radiator, and the batteries?', 'What will be done with the plastic materials in the vehicles?']\",\"question\":\"New technologies have allowed automobile manufacturers to market fuel-efficient hybrid cars, but disposal of hybrid cars at the end of their service lives presents a problem. Which question represents a disposal problem unique to hybrid vehicles?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/allenai/ai2_arc","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}