{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3fa6512c-d71c-59e6-be6b-3aa30c6bedae","task_key":"default--test--7025","task_revision_id":"1","upstream_id":"7025","short_description":"At 25 cents apiece, Mr.Krinskysells 100 chocolate bars per week. If he drops his…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Elastic', 'Elastic at 25 cents, Inelastic at 20 cents', 'Inelastic', 'Perfectly Elastic', 'Perfectly Elastic at 20 cents, Inelastic at 25 cents', 'Relatively Elastic', 'Unit Elastic', 'Unit Inelastic', 'Perfectly Inelastic']\",\"question\":\"At 25 cents apiece, Mr.Krinskysells 100 chocolate bars per week. If he drops his price to 20 cents, his weekly sales will increase to 110 bars. Is the demand for chocolate bars elastic or inelastic?\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}