{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1bedace7-6de3-5add-a688-97ca775c2920","task_key":"default--test--2537","task_revision_id":"1","upstream_id":"2537","short_description":"Jared thinks that going to psychology lecture classes is a waste of time and…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['just taking tests', 'going to lecture classes, reading the review book, and watching \\\"Discovering Psychology\\\"', 'going to lecture classes and watching \\\"Discovering Psychology\\\"', 'not going to lecture classes, reading the review book, and watching \\\"Discovering Psychology\\\"', 'going to lecture classes', 'reading the review book', 'watching \\\"Discovering Psychology\\\"', 'the review book and watching \\\"Discovering Psychology\\\"', 'not going to lecture classes and just taking tests', 'the tests scores']\",\"question\":\"Jared thinks that going to psychology lecture classes is a waste of time and that you can do just as well in the course if you just read the review book, watch \\\"Discovering Psychology,\\\" and take the tests. He decides to test his hypothesis with an experiment. The independent variable in his experiment is\",\"src\":\"ori_mmlu-high_school_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}