{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7aff28f0-c4cb-558c-9c64-f0b998fd181a","task_key":"default--test--8532","task_revision_id":"1","upstream_id":"8532","short_description":"A mass of $0.25 \\mathrm{~kg}$ is dropped from rest in a medium offering a…","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"['15.35 m/s', '16.10 m/s', '8.50 m/s', '13.45 m/s', '7.92 m/s', '9.63 m/s', '10.00 m/s', '11.58 m/s', '12.75 m/s', '14.20 m/s']\",\"question\":\"A mass of $0.25 \\\\mathrm{~kg}$ is dropped from rest in a medium offering a resistance of $0.2|v|$, where $v$ is measured in $\\\\mathrm{m} / \\\\mathrm{s}$.\\nIf the mass is dropped from a height of $30 \\\\mathrm{~m}$, find its velocity when it hits the ground.\",\"src\":\"scibench-diff\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}