{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ccea188c-e055-5313-a9e8-e4bbd87713ae","task_key":"default--test--2356","task_revision_id":"1","upstream_id":"2356","short_description":"We know less about why certain stimuli are attractive to humansthan we do about…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Attractiveness can be precisely measured and quantified in all individuals.', 'We fully understand the factors that determine human attractiveness.', 'Attractiveness is a trait that humans are born understanding completely.', 'Human attractiveness is determined by a single genetic marker.', 'The principles of attractiveness in humans have been fully mapped out by neuroscience.', 'Attractiveness is universally consistent across all human cultures.', 'The attractiveness of stimuli in humans is only influenced by visual cues.', 'Attractiveness is solely determined by physical appearance.', 'Attractiveness is a learned behavior.', 'Our knowledge about the specific characteristics that determine attractiveness remains on the level of intuition.']\",\"question\":\"We know less about why certain stimuli are attractive to humansthan we do about what is attractive to fish. Discuss thisstatement.\",\"src\":\"stemez-Psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}