{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"arc-challenge","formal_name":"ARC-Challenge","introduction":"ARC-Challenge is the subset of grade-school science questions that both a retrieval baseline and a word co-occurrence baseline answered incorrectly. It tests reasoning with knowledge rather than recall alone.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/allenai/ai2_arc","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b02335df-d33d-5cb9-8bf6-1938d1da7454","task_key":"ARC~2dChallenge--test--TIMSS~5f1995~5f8~5fR2","task_revision_id":"1","upstream_id":"TIMSS_1995_8_R2","short_description":"When white light shines on Peter's shirt, the shirt looks blue. Why does the…","config":"ARC-Challenge","split":"test","body":"{\"choices\":\"['It absorbs all the white light and turns most of it into blue light.', 'It reflects the blue part of the light and absorbs most of the rest.', 'It absorbs only the blue part of the light.', 'It gives off its own blue light.']\",\"question\":\"When white light shines on Peter's shirt, the shirt looks blue. Why does the shirt look blue?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/allenai/ai2_arc","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}