{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"drop","formal_name":"DROP","introduction":"DROP is a reading-comprehension benchmark whose questions require discrete operations over the paragraph — addition, subtraction, comparison, counting, sorting. It is built so that extracting a span cannot answer them.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/ucinlp/drop","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a365e649-937a-5215-8fb6-48518b49004f","task_key":"default--validation--7b25f67c~2d71fc~2d4926~2dba60~2dbe5c7d985a75","task_revision_id":"1","upstream_id":"7b25f67c-71fc-4926-ba60-be5c7d985a75","short_description":"Which console had sold the most in 1986?","config":"default","split":"validation","body":"{\"passage\":\"By the end of 1986, Computer Entertainer claimed the Atari 7800 had sold 100,000 consoles in the United States, less than the Sega Master Systems 125,000 and the Nintendo Entertainment Systems 1.1 million. According to Atari, due to manufacturing problems, it only managed to produce and sell 100,000 units by 1986, including units that had been in a warehouse since 1984. A common complaint in 1986 was a lack of games, including a gap of months between new releases (Galagas release in August was followed by Xevious in November). By the end of 1986, the 7800 had 10 games, compared to Segas 20 and Nintendos 36; nine of the NES games were third-party, whereas the 7800 and Master System had no third-party games. A reason cited for the lack of third-party interest in the 7800 was its small 100,000 install base and low market penetration.\",\"question\":\"Which console had sold the most in 1986?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/ucinlp/drop","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}