{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3472a5eb-563e-5f53-a4c4-c663658485e2","task_key":"default--test--1829","task_revision_id":"1","upstream_id":"1829","short_description":"A farmer has a large field where he grows corn. He hires a crop-duster to spray…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['neither the crop-duster nor the construction company is liable, because neither is a \\\"but for\\\" cause of the harm.', \\\"both the crop-duster and the construction company are liable, because each engaged in a negligent action that was sufficient to destroy the farmer's corn crop.\\\", \\\"only the crop-duster is liable, because its negligence occurred before the construction company's negligence.\\\", \\\"both the crop-duster and the construction company are liable, because they both contributed to the loss of the farmer's corn crop.\\\", 'neither the crop-duster nor the construction company is liable, because the farmer did not have insurance for his corn crop.', 'neither the crop-duster nor the construction company is liable, because the farmer failed to protect his field.', 'only the construction company is liable, because the fire caused more damage than the herbicide.', \\\"only the crop-duster is liable, because the construction company's fire was an unforeseeable event.\\\", \\\"only the construction company is liable, because its negligence would have destroyed the farmer's corn crop no matter what the crop- duster did.\\\", 'the crop-duster is liable for the value of the corn crop and the construction company is liable for the damage to the field.']\",\"question\":\"A farmer has a large field where he grows corn. He hires a crop-duster to spray the field, but the crop- duster mistakenly sprays a strong herbicide, rather than a pesticide, and kills all the corn plants in the field. Several days later, a construction company building a house near the farmer's field negligently hits a gas line and starts a fire that burns everything in the area, including the farmer's field. If the farmer brings negligence claims against the crop-duster and the construction company in order to recover damages for the destruction of his corn crop, the most likely result is\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}