{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1ec80961-d218-5d7e-a2ad-1fd2cb565d6e","task_key":"default--test--1805","task_revision_id":"1","upstream_id":"1805","short_description":"On August 1, a mother hired a photographer to take a series of photographs of…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['By entering into the contract as written, the photographer assumed the risk that the mother would fail to cooperate in arranging for the taking of the photographs.', 'The mother was under an implied duty to act in good faith to reasonably cooperate with the photographer in arranging for the taking of the photographs.', 'The mother had a legal obligation to ensure the two-year-old was cooperative during the photography session.', \\\"The mother's failure to cooperate would result in a full refund of the $200 payment.\\\", \\\"An implied condition of the photographer's duty of performance was that the mother reasonably cooperate in arranging for the taking of the photographs.\\\", \\\"The mother's refusal to cooperate reasonably in arranging for the taking of the photographs would excuse the photographer from further obligation under the contract and also give the photographer a right of action against the mother for breach of contract.\\\", 'The photographer had an implied duty to ensure the photographs were of a professional quality.', 'The photographer had a right to cancel the contract if the mother was not cooperating in arranging for the taking of the photographs.', 'The photographer was under an obligation to take more than six photographs if the mother was not satisfied with the initial results.']\",\"question\":\"On August 1, a mother hired a photographer to take a series of photographs of her two-year-old daughter. The written contract called for the photographer \\\"to produce a minimum of six acceptable photographs by August 15. \\\" In exchange, the photographer was to be paid $200. Which of the following is the LEAST accurate statement with regard to the photographer's taking of the photographs?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}