{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e95c2b48-7a6c-566b-8936-844996af3fb5","task_key":"default--test--12153","task_revision_id":"1","upstream_id":"12153","short_description":"Calculate the critical angle for an electromagnetic wave passingfrom the…","config":"default","split":"test","body":"{\"category\":\"engineering\",\"options\":\"['9 degrees for distilled water, 81.1 degrees for glass', '6.3 degrees for distilled water, 19.4 degrees for glass', '81.1 degrees for distilled water, 9 degrees for glass', '73 degrees for distilled water, 30 degrees for glass', '20 degrees for distilled water, 6.3 degrees for glass', '48.6 degrees for distilled water, 42.1 degrees for glass', '25 degrees for distilled water, 89 degrees for glass', '33 degrees for distilled water, 14 degrees for glass', '13.2 degrees for distilled water, 25.8 degrees for glass', '58 degrees for distilled water, 22 degrees for glass']\",\"question\":\"Calculate the critical angle for an electromagnetic wave passingfrom the following dielectrics into air. Material\\\\epsilon/\\\\epsilon_0(Ratio of permittivity to that of air) Distilled water81.1 Glass9\",\"src\":\"stemez-Electromagnetics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}