{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3b23f63f-1520-5848-8b0e-bbb8a81d01c5","task_key":"default--test--1017","task_revision_id":"1","upstream_id":"1017","short_description":"On February 1, a man conveys his orchard to a gardener, and the gardener duly…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['The gardener.', 'The investor.', 'The friend.', 'The landscaper and the friend jointly.', 'None of the parties, the original man reclaims the property.', 'The friend and the investor jointly.', 'The landscaper.', \\\"The investor's agent.\\\", 'The jurisdiction in which the property is located.', \\\"The gardener's friend.\\\"]\",\"question\":\"On February 1, a man conveys his orchard to a gardener, and the gardener duly records the conveyance. The following day, the gardener conveys the property to a landscaper; she does not record her deed. Then on February 4, the gardener executes an identical conveyance of the orchard to a friend. The friend gives the gardener a check for $100,000 for the property and records the conveyance, even though he has actual knowledge of the prior conveyance to the landscaper. The landscaper, however, records her deed on February 6. The friend then conveys his interest in the farm to an investor, who gives a purchase price of $115,000 to the friend. On February 5, the investor purchases the farm without notice of the conveyance to the landscaper and duly records the deed. Suppose that the jurisdiction in which the property is located has a pure race statute as their deed recordation law. Under these circumstances, which of the aforementioned parties would ultimately prevail?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}