{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2aa082b2-a7aa-5f18-aa91-4c2a73fc3742","task_key":"default--test--1699","task_revision_id":"1","upstream_id":"1699","short_description":"A circus operates in a rural part of the county. It is the circus's practice to…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"[\\\"not recover, because the adjuster's actions were part of his job duties.\\\", \\\"recover, because the adjuster's actions constituted harassment.\\\", 'not recover, because the parents took the advice of a neighbor and consulted an attorney.', 'not recover, because the parents did not suffer emotional distress that was severe.', \\\"not recover, because the parents eventually received a $250,000 settlement from the adjuster's insurance company.\\\", 'recover, because the adjuster provided false information to the parents.', \\\"not recover, because the adjuster's actions did not directly cause the boy's injury.\\\"]\",\"question\":\"A circus operates in a rural part of the county. It is the circus's practice to finish each day's entertainment by discharging a so-called aerial bomb into the sky. After exploding, the aerial bomb emits a spectacular rainbow fireworks display that is seen for miles around. One afternoon, a 10-year-old boy and a few friends went to the fairgrounds to see the circus. After paying their admission, they were about to enter the \\\"big top\\\" when the boy came upon an aerial bomb lying on the ground. Ignorant of what the object really was, but in an exploratory mood, the boy applied a match to the fuse of the fireworks device. It exploded and seriously injured the boy. After the accident, and before the parents of the boy had retained a lawyer, the adjuster for the circus's insurance carrier contacted the parents several times, trying to negotiate a settlement of their claim against the circus. On each occasion, the adjuster told the parents there was no need for them to hire an attorney because the applicable civil code did not provide for recovery in such a situation. The adjuster was aware that this information was blatantly false. He also warned the parents that unless they accepted his offered settlement of $5,000, they would receive nothing. For over a month, the adjuster continued to harass the parents. Outraged by the adjuster's actions, the parents sought the advice of a neighbor who recommended that they consult an attorney. The parents went ahead and retained a lawyer who subsequently negotiated a $250,000 settlement with the adjuster's insurance company for the claims relating to the boy's injury from the aerial bomb. If the parents assert a claim against the adjuster to recover damages for infliction of emotional distress, they will\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}