{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"764f8d20-988e-55c0-a1df-5cbf7db9a10c","task_key":"default--test--9507","task_revision_id":"1","upstream_id":"9507","short_description":"Two identical wires are stretched by the same tension of 100 N, and each emits a…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['5 cycles per second', '1 cycle per second', '0.5 cycle per second', '4 cycles per second', '2.5 cycles per second', '2 cycles per second', '0.2 cycle per second', '0.8 cycle per second', '3 cycles per second', '1.5 cycles per second']\",\"question\":\"Two identical wires are stretched by the same tension of 100 N, and each emits a note of frequency 200 cycles\\\\textbullets^\\\\rule{1em}{1pt}1. The tension in one wire is increased by 1 N. Calculate the number of beats heard per second when the wires are plucked.\",\"src\":\"stemez-Physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}