{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"405c5fa8-1bd9-5c24-b6bf-69368633bc7c","task_key":"default--test--10422","task_revision_id":"1","upstream_id":"10422","short_description":"If your training loss increases with number of epochs, which of the following…","config":"default","split":"test","body":"{\"category\":\"computer science\",\"options\":\"['Regularization is too high and model is underfitting', 'The number of epochs is too high', 'Step size is too small', 'Step size is too large', 'The number of epochs is too low', 'Regularization is too low and model is overfitting', 'The model is not complex enough', 'The data is not shuffled before each epoch', 'The model is too complex', 'The model is perfect, no issues']\",\"question\":\"If your training loss increases with number of epochs, which of the following could be a possible issue with the learning process?\",\"src\":\"ori_mmlu-machine_learning\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}