{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e22b8c26-e645-544d-b8d7-2b0d6e9f1c64","task_key":"default--test--1646","task_revision_id":"1","upstream_id":"1646","short_description":"On February 1, a woman conveys her farm to a man, and the man duly records the…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['The man and the farmer.', 'The sister and the friend.', 'The friend.', 'The man.', 'None of the above.', 'The woman.', 'The sister.', 'Both the man and the friend.', 'The farmer.', 'Both the sister and the farmer.']\",\"question\":\"On February 1, a woman conveys her farm to a man, and the man duly records the conveyance. The following day, the man conveys the property to his sister; she does not record her deed. Then on February 4, the man executes an identical conveyance of the farm to a friend. The friend gives the man a check for $100,000 for the property and records the conveyance, even though he has actual knowledge of the prior conveyance to the sister. The sister, however, records her deed on February 6. The friend then conveys his interest in the farm to a farmer, who gives a purchase price of $115,000 to the friend. On February 5, the farmer purchases the farm without notice of the conveyance to the sister and duly records the deed. If the property in question was located in a state having a notice-type statute, which of the following parties would ultimately prevail?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}