Engadget has been testing and reviewing consumer tech since 2004. Our stories may include affiliate links; if you buy something through a link, we may earn a commission. Read more about how we evaluate products.
Google went down after traffic was routed through China and Russia
Google said it wasn't malicious, but the timing is odd.
Google's services went down for an hour yesterday after its IP addresses were routed way from normal paths to Nigeria, China and Russia. Google told Ars Technica it doubted the leak was malicious, despite the fact that government-owned China Telecom was recently caught routing Western carrier traffic through mainland China. Some of Google's most sensitive data, including its corporate WAN infrastructure and VPN, were reportedly redirected.
The problem started when a carrier in Lagos, Nigeria improperly declared its own system as the correct route to several hundred IP prefixes belonging to Google. China Telecom accepted the route (also improperly) and declared it worldwide. That in turn was picked up by Russia's Transtelecom and other large ISP services. Later on, the same Nigerian carrier made a second incorrect IP declaration that sent Google partner Cloudflare's IP addresses on a similar joyride.
This incident at a minimum caused a massive denial of service to G Suite and Google Search. However, this also put valuable Google traffic in the hands of ISPs in countries with a long history of Internet surveillance. Overall ThousandEyes detected over 180 prefixes affected by this route leak, which covers a vast scope of Google services.
Cloudflare CEO Matthew Prince told Ars that the nature of the misdirection points to a "big, ugly screw-up" rather than anything malicious. "If there was something nefarious afoot, there would have been a lot more direct, and potentially less disruptive/detectable ways to reroute traffic," he said. Instead, it might have been related to recent network meetings in Nigeria. "While setting up a new interconnection, the Nigerian ISP almost certainly inadvertently leaked the routing information to China Telecom who then leaked it out to the rest of the world," said Prince.
Google said that its services weren't compromised because almost all of its traffic is encrypted. (Facebook also experienced a rare outage yesterday that was reportedly unrelated.) It's a reminder of how sensitive global internet protocols heavily rely on trust, something that's lacking in today's climate of online spying, election hacking, cryptocurrency theft and other major issues.
While Google wasn't too concerned about foul play, it was still a major outage that mostly affected users of its paid businesses, rather than consumer products. "This incident at a minimum caused a massive denial of service to G Suite and Google Search," said security research firm Thousand Eyes. "However, this also put valuable Google traffic in the hands of ISPs in countries with a long history of Internet surveillance."
Update 11/13/2018 4:13 PM ET: Google has provided Engadget with the following statement:
We're aware that a portion of internet traffic was affected by incorrect routing of IP addresses, and access to some Google services was impacted. The root cause of the issue was external to Google and there was no compromise of Google services.
