{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4954cd1f-963a-50cd-ac83-c75dd72f1eaf","task_key":"default--test--2807","task_revision_id":"1","upstream_id":"2807","short_description":"In reference to a segment of DNA, which of the following molecules contains the…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['the tRNA transcript from the original DNA', 'a single strand of the original DNA segment after a substitution mutation', 'the final processed mRNA made from the original DNA', 'the primary RNA transcript (after splicing) from the original DNA', 'a single strand of the original DNA segment after a duplication mutation', 'the primary RNA transcript (before splicing) from the original DNA', 'a single strand of the original DNA segment after a deletion mutation', 'a single strand of the original DNA segment', 'a single strand of complementary DNA (cDNA) made from the original DNA', 'a single strand of the original DNA segment after a point mutation']\",\"question\":\"In reference to a segment of DNA, which of the following molecules contains the fewest number of nucleotides?\",\"src\":\"ori_mmlu-high_school_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}