{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cca21d01-3f58-5d41-a6ed-33708aa4d402","task_key":"default--test--1717","task_revision_id":"1","upstream_id":"1717","short_description":"A professor was the record owner in fee simple absolute of a 30-acre tract of…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['in the bank until the mortgage is paid, then in the co-worker and the uncle.', 'half in the co-worker and half in the bank, with both subject to the mortgage.', 'entirely in the uncle, subject to the mortgage.', 'entirely in the uncle, free and clear of the mortgage.', 'half in the co-worker and half in the uncle, with both subject to the mortgage.', \\\"in the co-worker and the uncle, with the co-worker's portion free of the mortgage and the uncle's portion subject to it.\\\", 'half in the co-worker, free of the mortgage, and half in the uncle, subject to the mortgage.', 'in the co-worker, with the entire interest subject to the mortgage.', 'half in the co-worker, subject to the mortgage, and half in the uncle, free of the mortgage.']\",\"question\":\"A professor was the record owner in fee simple absolute of a 30-acre tract of land located in a small town. The professor made a conveyance thereof in these words, \\\"I hereby convey my 30-acre tract to both my friend and my co-worker as joint tenants with right of survivorship. \\\"Two years after making the conveyance to the friend and the co-worker, the friend executed a mortgage on the 30-acre tract of land to a bank to secure a loan. One year after executing the mortgage, the friend died intestate, leaving his uncle as his only heir. At the time of the friend's death, the indebtedness had not been paid to the bank. The jurisdiction in which the 30-acre tract of land is located recognizes a title theory of mortgages. In an appropriate action, the court should determine that title to the 30-acre tract of land is vested\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}