{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b631addc-25b2-530f-90ef-c4b2e861f712","task_key":"default--test--11000","task_revision_id":"1","upstream_id":"11000","short_description":"In response to the argument that embryos are not persons because they do not…","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"[\\\"it doesn't make sense to say that a person exercises higher mental capacities.\\\", 'mental capacities should not be a factor in determining personhood.', 'the ability to exercise higher mental capacities could be considered a potential, not an actuality.', 'to have killed a human embryos that developed into you would have been the same thing as killing you.', 'embryos actually do exercise higher mental capacities.', 'higher mental capacities are not the only defining characteristics of a person.', 'they dispute the notion that higher mental capacities are a requirement for personhood.', 'the argument is irrelevant as they believe life begins at conception.', 'embryos are potential persons and therefore should have the same rights.', 'the argument is sound, but its validity fails due to a common logical fallacy.']\",\"question\":\" In response to the argument that embryos are not persons because they do not exercise higher mental capacities, Lee and George argue that\",\"src\":\"ori_mmlu-moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}