{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3637c57f-2b20-5dbd-a195-26dd625425cb","task_key":"default--test--246","task_revision_id":"3","upstream_id":"246","short_description":"Costs for producing one widget: Materials:2 parts (a) $25.00 per part Expenses…","config":"default","split":"test","body":"{\"category\":\"business\",\"options\":\"['Each widget requires $50.00 in labor for 10 operations', 'Material costs represent the smallest fraction of the total production cost', 'output is labor intensive', 'material costs are low', 'Expenses are greater than the combined costs of materials and labor', 'The selling price should be $160 per widget', 'The most significant cost in producing a widget is the expense fee', 'labor rates are excessive', 'The total cost of production is less than $100 per widget', 'There are 10 operations involved in labor, each at $10.00 per operation']\",\"question\":\"Costs for producing one widget: Materials:2 parts (a) $25.00 per part Expenses :$10.00 Labor :20 operations (a) $5.00 per operation Based on the above information, which is the only valid conclusion? (A) The selling price should be $160 per widget. (B) labor rates are excessive (C) output is labor intensive (D) material costs are low\",\"src\":\"stemez-Business\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}