{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6639dd0d-b7eb-5c58-aec4-9dc95fab93cf","task_key":"default--test--7135","task_revision_id":"1","upstream_id":"7135","short_description":"Define poverty.","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Poverty is the lack of resources to secure basic life necessities.', 'Poverty is the condition where a person has excessive wealth and resources.', 'Poverty is the inability to afford a good standard of living.', 'Poverty is the condition of having too little money to fulfill basic needs as society currently defines those needs.', 'Poverty is the inability to purchase luxury items and services.', 'Poverty is the state of living in a rural area without urban amenities.', 'Poverty is the temporary lack of employment.', 'Poverty is the state of being inferior in quality or insufficient in amount.', 'Poverty is the condition of having an income that is less than the median income of a country.', 'Poverty is exclusively the state of not having access to education.']\",\"question\":\"Define poverty.\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}