Astronomers used to think that an asteroid big enough to do serious damage to a city came along about once a century. So far no cities have been destroyed, and the most damaging one was the one that exploded over Chelyabinsk in 2013. There was the Tunguska event in Siberia in the previous century, but that was a sparsely inhabited area. Remember only about thirty percent of Earth's surface is land. And of that there are large deserts etc., so such an explosion wouldn't be noticed. Until recently.
The Nuclear Test Ban Treat Organization operates a network of sensors that monitor Earth to listen for nuclear detonations. What it found between 2000 and 2013 was 26 explosions that were caused by asteroids. The energy released was between 1-600 kilotons. (The Hiroshima bomb was 15 kilotons.) These data show that we encounter many more large space objects than we had thought, and that they arrive undetected.