{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4c5f6905-8051-5621-8a56-9571c8cbaa0d","task_key":"default--test--3494","task_revision_id":"1","upstream_id":"3494","short_description":"Brood parasites such as the cuckoo successfully trick other species of birds…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['the host birds learn from the experience and are less likely to be duped in the future', \\\"on average, the host birds' response allows them to rear their own young efficiently by feeding only those who indicate they are hungry\\\", 'the cuckoo fledglings share some of their genes with the host birds', 'the cuckoo fledglings contribute to the survival of the host bird species by increasing genetic diversity', 'the maximum fitness of the duped bird is not compromised when the bird rears an interloper of another species', 'the maximum fitness of the cuckoo would then be reduced', \\\"on average, little energy is spent on rearing a fledgling bird, whether it is an interloper or one's own\\\", 'the cuckoo fledglings help to protect the nest from predators', 'the genes that allow the host birds to be duped are necessary for the survival of the cuckoo species']\",\"question\":\"Brood parasites such as the cuckoo successfully trick other species of birds into rearing their young by exploiting the host birds' instinctive response to the loud begging cues of a fledgling in their nest. The genes that allow the host bird species to be duped into rearing the cuckoo fledglings to the detriment of their own offspring most likely remain in the gene pool of the population because\",\"src\":\"ori_mmlu-college_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}