{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f06ed482-b9ca-5691-9b86-d1ada2b9de41","task_key":"default--test--2973","task_revision_id":"1","upstream_id":"2973","short_description":"Why would one call a flock of birds a…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['Birds form societies to find food', 'A flock of birds can be considered an elementary form of society called a motion group. They communicate with one another in order to stay together as they move from place to place. The cooperative behavior enables them to more efficiently detect and avoid predators.', 'A flock of birds is called a society because they display human-like social structures.', 'Birds are classified as a society because they share a common language and culture.', 'Birds form societies to perform complex problem-solving tasks as a group.', 'Birds form societies to migrate together to the same location every year.', 'Birds are considered a society because they are governed by a single leader.', 'Birds form societies primarily for mating purposes.', 'Birds form societies for social interaction', 'Birds form societies to build nests together']\",\"question\":\"Why would one call a flock of birds a \\\\textquotedblleftsociety\\\\textquotedblright?\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}