Google cloud falls over after routing error, strives to remove manual link activation

November 30, 2015 Off By David
Object Storage

Grazed from CloudTech. Author: James Bourne.

Google Compute Engine went down for approximately 70 minutes last week, the company has confirmed, making certain Internet destinations unreachable from the europe-west1 region during that time. The issue first came to light at 1326 PST on November 23 with a status update, before a further missive at 1432 confirming the problems should have been resolved.

Four days later, Google explained what exactly went wrong. At 1151 PST on November 23, Google engineers activated a new peering link – with an unnamed provider who Google says it works with extensively – but during the activation, the providers’ estimations of how much capacity the link could take differed wildly from actual performance…

As a result, traffic was dropped with the majority of affected destination addresses coming from eastern Europe and the Middle East. The reason for the oversight, Google notes, was due to an ‘unrelated failure’ which meant safety checks as part of the automation process for peering links were not performed. The search giant says it will change procedure to prevent manual link activation following the downtime…

Read more from the source @ http://www.cloudcomputing-news.net/news/2015/nov/30/google-cloud-falls-over-after-routing-error-strives-remove-manual-link-activation/