{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8a75dc34-1cb3-5b67-88eb-df04b3596565","task_key":"default--test--9511","task_revision_id":"1","upstream_id":"9511","short_description":"To what tension must a brass wire of cross-sectional area 10^\\rule{1em}{1pt}2…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['Depends on the frequency of the waves', 'Depends on the material', 'Depends on the tension applied', 'Depends on the length of the wire', 'Depends on the temperature of the wire', 'Realizable if the wire is alloyed with another metal', 'Only realizable in a vacuum', 'Physically realizable', 'Physically unrealizable', 'Realizable at a specific tension that matches the speed of sound in brass']\",\"question\":\"To what tension must a brass wire of cross-sectional area 10^\\\\rule{1em}{1pt}2 cm^2 be subjected so that the speed of longitudinal and transverse waves along it may be the same? Young's modulus for brass is 9.1 x 10^11 dynes\\\\textbulletcm^\\\\rule{1em}{1pt}2. Is this situation physically realizable?\",\"src\":\"stemez-Physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}