Microsoft Promises Service Credits To Office 365 Customers Hit By Outages6:55 PM EST Thu. Nov. 15, 2012
Microsoft will be issuing service credits to Office 365 customers who were hit by email service outages last week and again earlier this week. The company also says the problems that caused the outages have been resolved and a review is underway to prevent a re-occurrence.
"All of us in the Office 365 team and at Microsoft appreciate the serious responsibility we have as a service provider to you, and we know that any issue with the service is a disruption to your business -- that's not acceptable," wrote Rajesh Jha, corporate vice president of the Microsoft Office Division, in a blog post about the outages.
"I want to assure you that we are investing the time and resources required to ensure we are living up to your -- and our own -- expectations for a quality service experience every day."
Customers in North and South America experienced an eight-hour outage of their Office 365 Exchange Online service on Nov. 8, lasting from 11:24 a.m. to 7:25 p.m. PST, resulting in "prolonged mail flow delays" for customers, Jha said. The second email service outage, which lasted more than five hours on Tuesday, began at 9:08 a.m. and was resolved at 2:10 p.m. PST.
Jha's blog post offers detailed descriptions for the causes of both outages.
The Nov. 8 outage began when an anti-virus engine identified a virus being sent to subscribers, but "started to exhibit a lot of latency even as it handled the messages," the blog stated. Compounding the problem, the service was configured in such a way that it allowed too many retries and too long of a timeout for those messages, causing "a significant backlog of valid email messages."
This week's outage was caused by "a combination of issues related to maintenance, network element failures, and increased load on the service," Jha said. An Office 365 team was performing "planned, non-impacting network maintenance" and they shifted some workload out of the data center where the maintenance work was being done. That, combined with a failure of some networking technology and an increased load of customers subscribing to the service, "caused customer access to email services to be degraded for an extended period of time."
Only email services were affected, according to Jha, and no other Microsoft services were impacted.
NEXT: Customers Praise, Criticize Microsoft's Response
Additionally, Microsoft's Jha made a formal apology in his blog post, extending a service credit to those customers who were impacted by the outage.
"I'd like to apologize to you, our customers and partners, for the obvious inconveniences these issues caused," Jha wrote. "We know that email is a critical part of your business communication, and my team and I fully recognize our responsibility as your partner and service provider. We will provide a post mortem, and will also provide additional updates on how our service level agreement (SLA) was impacted. We will be proactively issuing a service credit to our impacted customers."
Comments posted to Jha's blog are mixed, with some praising Microsoft for detailing the cause of the outages. "I truly appreciate the transparency reflected in this post," wrote one subscriber. "I hope it is perpetuated throughout Microsoft. O365 is a journey that we are in together and this transparency makes me as a customer feel more like a partner in this journey."
Others aren't as forgiving. "I appreciate the fact that you have provided an update with an actual explanation," a subscriber wrote. "However, that doesn't excuse the terrible communication and process failure that occurred during the outage. In addition to addressing the technical issues that caused both of these outages, Microsoft also needs to address the monitoring and communications processes."
PUBLISHED NOV. 15, 2012