{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8848bcee-5347-5830-abe3-5eb331552ab7","task_key":"default--test--2243","task_revision_id":"1","upstream_id":"2243","short_description":"What is the Theory of SignalDetectabilityand how is it different from the more…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['TSD and the classical psychophysical view are identical in their approach to detecting stimuli, with no significant differences.', \\\"TSD considers only the subject's ability to detect a stimulus\\\", 'The classical view relies on signal intensity alone for detection, while TSD introduces the idea of decision criteria set by the observer.', \\\"The classical view incorporates both the subject's ability and motivation to detect a stimulus, while TSD focuses solely on the ability.\\\", \\\"Classical view takes into account the subject's motivation to detect a stimulus\\\", 'TSD suggests that detection is purely a matter of chance, whereas the classical view is based on probability theory.', \\\"TSD is based on the concept of 'thresholds' being fixed values, unlike the classical view which sees them as variable.\\\", 'TSD is a simpler approach than the classical psychophysical view', \\\"The Theory of SignalDetectability(TSD) maintains that the classical view of the 'threshold' is an oversimplification. It argues that a subject's report of a signal presence tells two things: the ability to detect the stimulus and the motivation to detect it. The classical view only considers discrimination/detection/recognition ability.\\\"]\",\"question\":\"What is the Theory of SignalDetectabilityand how is it different from the more classical psychophysical view?\",\"src\":\"stemez-Psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}