{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"17a7d3dd-28a0-54c7-baf7-d4a2625d9c20","task_key":"default--test--4648","task_revision_id":"1","upstream_id":"4648","short_description":"Calculate the pH of (a) a 0.5 M solution with respect to CH_3COOH and CH_3COONa;…","config":"default","split":"test","body":"{\"category\":\"chemistry\",\"options\":\"['4.70, 4.60', '4.85, 4.50', '4.76, 4.65', '4.65, 4.85', '4.85, 4.76', '5.00, 4.90', '4.58, 4.76', '4.90, 4.76', '4.65, 4.76', '4.76, 4.58']\",\"question\":\"Calculate the pH of (a) a 0.5 M solution with respect to CH_3COOH and CH_3COONa; (b) the same solution after 0.1 mole HCl per liter has been added to it. Assume that the volume is unchanged. K_a = 1.75 × 10^-5.\",\"src\":\"stemez-Chemistry\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}