{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cda67701-2220-5bfd-890b-6805ccc5c223","task_key":"default--test--3168","task_revision_id":"1","upstream_id":"3168","short_description":"When estimating the total number of Calories required by a personin a day, we…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['Caloric requirements are only influenced by the environmental temperature, not by sex or activity level.', 'Temperature has no effect on caloric requirements.', \\\"Each person's Caloric requirement differs according to sex, amount of activity and environmental temperature.\\\", 'Caloric requirements are determined by age and sex only, without taking into account activity level or temperature.', 'Only the sex of the person matters for caloric requirements.', 'Caloric requirements are determined by a fixed formula that applies universally, regardless of individual differences.', 'Caloric requirements decrease with increased activity and are independent of sex or environmental factors.', 'Caloric requirement is the same for everyone.', 'Caloric requirements increase only with the amount of activity and are unaffected by sex or temperature.', \\\"Caloric requirements are solely based on an individual's weight and height, with no consideration for sex, activity, or temperature.\\\"]\",\"question\":\"When estimating the total number of Calories required by a personin a day, we consider such factors as sex, amount of activity, and the temperature in which the person lives. Why arethese factors necessary?\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}