{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fd4ef2e3-d84b-54e5-b403-be9decd591aa","task_key":"default--test--2779","task_revision_id":"1","upstream_id":"2779","short_description":"Why might a researcher use a variable ratio of reinforcement rather than a fixed…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Fixed ratio schedules of reinforcements are more time intensive.', 'Fixed ratio schedules of reinforcement allow for easier data collection.', 'Variable ratio schedules of reinforcement are less effective but easier to implement.', 'Variable ratio schedules of reinforcements avoid problems such as generalization and the Premack principle.', 'Variable ratio schedules of reinforcements are less susceptible to habituation.', 'Fixed ratio schedules of reinforcement are less predictable, thus more effective.', 'Fixed ratio schedules of reinforcement are more resistant to extinction than variable schedules.', 'Variable ratio schedules of reinforcement are more resistant to extinction than fixed schedules.', 'Variable ratio schedules of reinforcements produce results more quickly.']\",\"question\":\"Why might a researcher use a variable ratio of reinforcement rather than a fixed ratio?\",\"src\":\"ori_mmlu-high_school_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}