{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"471261fc-d148-531a-a8ae-651f33951ceb","task_key":"default--test--557","task_revision_id":"1","upstream_id":"557","short_description":"A contractor has found through experience that the low bid for a job (excluding…","config":"default","split":"test","body":"{\"category\":\"business\",\"options\":\"['k = (7c / 4)', 'k = (3c / 2)', 'k = (3c / 4)', 'k = (c / 3)', 'k = (4c / 3)', 'k = c', 'k = (2c / 3)', 'k = (5c / 4)', 'k = (5c / 3)', 'k = (c / 2)']\",\"question\":\"A contractor has found through experience that the low bid for a job (excluding his own bid) is a random variable that is uniformly dis-tributed, over the interval ( 3c / 4 , 2c) where c is the contractor's cost estimate (no profit or loss) of the job. If profit is defined as zero if the contractor does not get the job (his bid is greater than the low bid) and as the difference between his bid and the cost estimate c if he gets the job, what should he bid, in terms of c, in order to maximize his expected profit?\",\"src\":\"stemez-Business\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}