{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c4ec6025-bf12-578a-adf3-51897dfa6498","task_key":"default--test--10151","task_revision_id":"1","upstream_id":"10151","short_description":"An electrical transmission line is strung across a valley between two utility…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['6.0 × 10^2 N', '2.8 × 10^2 N', '2.4 × 10^2 N', '3.2 × 10^2 N', '1.8 × 10^2 N', '5.0 × 10^2 N', '3.8 × 10^2 N', '3.5 × 10^2 N', '4.0 × 10^2 N', '4.5 × 10^2 N']\",\"question\":\"An electrical transmission line is strung across a valley between two utility towers. The distance between the towers is 200 m, and the mass of the line is 40 kg. A lineman on one tower strikes the line and a wave travels to the other tower, is reflected, and is detected by the lineman when it returns. If the elapsed time measured by the lineman is 10 s, what is the tension force in the wire?\",\"src\":\"stemez-Physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}