{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"25229b0d-a8e1-5603-babe-57d053af69f7","task_key":"default--test--11066","task_revision_id":"1","upstream_id":"11066","short_description":"What are the two principal answers to whether life has any meaning for us?","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"[\\\"the pessimist'sview and the optimist'sview\\\", \\\"the existentialist's view and the nihilist's view\\\", 'the deterministic view and the free will view', 'the materialistic view and the spiritual view', \\\"the empiricist's view and the rationalist's view\\\", 'the external view and the internal view', 'the theistic view and the agnostics view', \\\"the monist's view and the dualist's view\\\", \\\"the relativist's view and the absolutist's view\\\", 'the religious view and the liberal view']\",\"question\":\"What are the two principal answers to whether life has any meaning for us?\",\"src\":\"ori_mmlu-philosophy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}