{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"41b0d9bd-b799-5319-a042-72123ccc8ca1","task_key":"default--test--11899","task_revision_id":"1","upstream_id":"11899","short_description":"The metal beaker of a dew-point apparatus is gradually cooled from room…","config":"default","split":"test","body":"{\"category\":\"engineering\",\"options\":\"['0.1987psia, 0.01489', '0.2563psia, 0.01089', '0.2890psia, 0.01123', '0.2756psia, 0.00987', '0.3125psia, 0.00765', '0.2105psia, 0.01357', '0.2654psia, 0.00942', '0.2304psia, 0.01234', '0.3750psia, 0.00658', '0.3452psia, 0.00876']\",\"question\":\"The metal beaker of a dew-point apparatus is gradually cooled from room temperature, 75°F. When the beaker temperature reaches 60°F, the moisture of the room air starts condensing on it. Assuming the room air to be at 14.7 psia, determine (a) the partial pressure of vapor, and (b) the parts by mass of vapor in the room air.\",\"src\":\"stemez-Thermodynamics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}