{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"219d1181-9fce-5a0e-8b6e-18b377672920","task_key":"default--test--2193","task_revision_id":"1","upstream_id":"2193","short_description":"In a family where all male members have been doctors for generations, Peter…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"[\\\"Peter's parents wanted him to continue the family legacy.\\\", \\\"Peter's parents' attitude can be explained by the threat to their self-structure, which contains the notion that to be of value, a male family member must be a doctor. This idea is incongruent with their love for their son, causing distress.\\\", \\\"Peter's parents believe that being a doctor is the only respectable profession in society.\\\", \\\"Peter's parents are afraid that Peter's choice indicates a rejection of family values.\\\", \\\"Peter's parents think Peter is not talented enough to be a successful artist.\\\", \\\"Peter's parents are worried about the social stigma associated with not following family traditions.\\\", \\\"Peter's parents are concerned about the financial stability of being an artist.\\\", \\\"Peter's parents are upset because they don't value the arts.\\\", \\\"Peter's parents are upset because they fear he will not be able to contribute to the family's reputation.\\\", \\\"Peter's parents assume that Peter's decision is a phase and he will eventually become a doctor.\\\"]\",\"question\":\"In a family where all male members have been doctors for generations, Peter decides to become an artist. Although his parentsclaim to love him they are very upset over his choice. How would Rogers explain this attitude shownby Peter's parents?\",\"src\":\"stemez-Psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}