{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8c0cc489-9f0e-5f58-b700-c8788b21deb3","task_key":"all--test--8c0cc489-9f0e-5f58-b700-c8788b21deb3","task_revision_id":"1","upstream_id":"","short_description":"A passage from Zora Neale Hurston's Their Eyes Were Watching God is as follows.…","config":"all","split":"test","body":"{\"choices\":\"[\\\"The sentences emphasize the weariness the sitters feel after a long day's work.\\\", 'The sentences are used to paint a picture of the way in which the sitters wish they spent their evenings.', 'The sentences are a vivid way of describing the ease and authority the sitters feel during the evening.', 'The sentences highlight the contrast between the feelings of the sitters and the feelings of the bossman.']\",\"question\":\"A passage from Zora Neale Hurston's Their Eyes Were Watching God is as follows. The people all saw her come because it was sundown. The sun was gone, but he had left his footprints in the sky. It was the time for sitting on porches beside the road. It was the time to hear things and talk. These sitters had been tongueless, earless, eyeless conveniences all day long. Mules and other brutes had occupied their skins. But now, the sun and the bossman were gone, so the skins felt powerful and human. They became lords of sounds and lesser things. They passed nations through their mouths. They sat in judgment. Which of the following is the best way of describing the last three sentences of the passage (\\\"They became . . . in judgment\\\") ?\",\"subject\":\"miscellaneous\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}