If you have experienced outage or faced with issues while using some of the Google products today then you were not alone. It happened to 10% of overall Google users around the world.
12 Google Services Experienced Outage Today
|Gmail||1/24/14 11:12 AM – 1/24/14 12:23 PM||71 minutes|
|Google Calendar||1/24/14 11:58 AM – 1/24/14 12:25 PM||27 minutes|
|Google Talk||1/24/14 11:58 AM – 1/24/14 12:25 PM||27 minutes|
|Google Drive||1/24/14 11:58 AM – 1/24/14 12:25 PM||27 minutes|
|Google Docs||1/24/14 11:58 AM – 1/24/14 12:25 PM||27 minutes|
|Google Sheets||1/24/14 11:58 AM – 1/24/14 12:25 PM||27 minutes|
|Google Slides||1/24/14 11:58 AM – 1/24/14 12:25 PM||27 minutes|
|Google Drawings||1/24/14 11:58 AM – 1/24/14 12:25 PM||27 minutes|
|Google Sites||1/24/14 11:58 AM – 1/24/14 12:25 PM||27 minutes|
|Google Groups||1/24/14 11:58 AM – 1/24/14 12:25 PM||27 minutes|
|Google+ Hangouts||1/24/14 11:58 AM – 1/24/14 12:51 PM||53 minutes|
|Google Voice||1/24/14 11:58 AM – 1/24/14 12:25 PM||27 minutes|
* Above data is according to information provided through Google Apps Dashboard
According to Google, All these outages were caused due to a software bug in an internal system that generates configuration information for google services/servers and replicates across its servers around the world. By the time the bug was identified, the misconfigured information was applied to servers that serve over 10% of Google product users.
Immediately the buggy system was taken down and that stopped the spread of the incorrect configuration. Google just has released an update through this blog post.
First of its kind outage of Gmail, one of the widely used Google product around the world has caused most panic of all. It was impossible to login to Gmail or read emails or do anything on the Gmail site and lasted for about an hour. Thanks God the issue was identified quickly and prevented from propagating further.
In his blog post, Ben Treynor VP Engineering at Google outlined steps they are taking in-order to prevent this from happening again
With services once again working normally, our work is now focused on (a) removing the source of failure that caused today’s outage, and (b) speeding up recovery when a problem does occur. We’ll be taking the following steps in the next few days:
1. Correcting the bug in the configuration generator to prevent recurrence, and auditing all other critical configuration generation systems to ensure they do not contain a similar bug.
2. Adding additional input validation checks for configurations, so that a bad configuration generated in the future will not result in service disruption.
3. Adding additional targeted monitoring to more quickly detect and diagnose the cause of service failure.
Google’s Apps Status Dashboard is a great way to know what’s going on with Google services when you see something not working as it supposed to be. You may also try those Google service website links (for e.g Gmail, Docs, etc) through other 3rd party check website status services to see if it is up or down for everyone.
Did you experience outage with Gmail or other Google Services today? Share your thoughts through Google+ comments below.