{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"60566676-051f-51f3-ac43-050c28da7e4f","task_key":"default--test--7569","task_revision_id":"1","upstream_id":"7569","short_description":"If firms are entering an industry that is monopolistically competitive, we would…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['the number of firms in the industry to decrease.', 'the market supply to become inelastic.', 'the market demand to become more elastic.', 'the market price of the product to decrease.', 'the demand for existing firms to become more elastic.', 'the demand for existing firms to become more inelastic.', 'the market supply to shift rightward.', 'the demand for existing firms to shift rightward.', 'the market price of the product to increase.', 'the demand for existing firms to shift leftward.']\",\"question\":\"If firms are entering an industry that is monopolistically competitive, we would expect\",\"src\":\"ori_mmlu-high_school_microeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}