{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9d7486c1-5793-5237-97b2-fac58027975c","task_key":"default--test--10488","task_revision_id":"1","upstream_id":"10488","short_description":"For a neural network, which one of these structural assumptions is the one that…","config":"default","split":"test","body":"{\"category\":\"computer science\",\"options\":\"['The number of input nodes', 'The type of loss function used', 'The type of optimization algorithm used', 'The number of hidden nodes', 'The initial choice of weights', 'The number of layers in the network', 'The choice of activation function', 'The use of a constant-term unit input', 'The learning rate', 'The number of output nodes']\",\"question\":\"For a neural network, which one of these structural assumptions is the one that most affects the trade-off between underfitting (i.e. a high bias model) and overfitting (i.e. a high variance model):\",\"src\":\"ori_mmlu-machine_learning\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}