{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"drop","formal_name":"DROP","introduction":"DROP is a reading-comprehension benchmark whose questions require discrete operations over the paragraph — addition, subtraction, comparison, counting, sorting. It is built so that extracting a span cannot answer them.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/ucinlp/drop","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"384c8aa6-8568-5f1a-9b42-f8837043c696","task_key":"default--validation--bd59a7c7~2d24c5~2d424c~2db576~2de1736454c70b","task_revision_id":"1","upstream_id":"bd59a7c7-24c5-424c-b576-e1736454c70b","short_description":"How many millions of pounds did the spending on trunk roads in Scotland go up…","config":"default","split":"validation","body":"{\"passage\":\"The road building programme was also expanded, with capital expenditure increased to 8% of GDP, \\\"the highest level achieved by any post-war government\\\". Central government expenditure on roads went up from £125 million in 1963/64 to £225 million in 1967/68, while a number of road safety regulations were introduced, covering seat belts, lorry driver's hours, car and lorry standards, and an experimental 70 mile per hour speed limit. In Scotland, spending on trunk roads went up from £6.8 million in 1963/64 to £15.5 million in 1966/67, while in Wales, spending on Welsh roads went up from £21.2 million in 1963/64 to £31.4 million in 1966/67.\",\"question\":\"How many millions of pounds did the spending on trunk roads in Scotland go up between 1963 and 1967?\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/ucinlp/drop","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}