{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f92af59e-fd3c-5596-8393-4727b99e26ae","task_key":"default--test--10625","task_revision_id":"1","upstream_id":"10625","short_description":"In systems with support for automatic memory management, a garbage collector…","config":"default","split":"test","body":"{\"category\":\"computer science\",\"options\":\"['Newly created objects awaiting assignment', 'Return addresses on the call stack', 'Objects referenced by CPU instructions', 'Temporary variables in the active procedures', 'Global variables of the program', 'Local variables on the call stack', 'Dynamically allocated objects on the heap', 'Static variables in the program', 'Actual parameters of the active procedures', 'Objects stored in registers']\",\"question\":\"In systems with support for automatic memory management, a garbage collector typically has the responsibility for reclaiming allocated memory objects whose contents cannot affect any future legal computation. Such objects are identified by determining that they cannot be reached from a root set. Which of the following is NOT part of the root set in a typical garbage collector?\",\"src\":\"ori_mmlu-college_computer_science\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}