Intuit’s hosted services for SMBs, including QuickBooks Online, QuickBooks Online Payroll and Intuit Payments Solutions, suffered a series of outages last week that left the software vendor’s partners and customers frustrated.
The service failures occurred Monday, Tuesday and Friday, although some businesses had trouble accessing the applications for much of the week.
"The outage has lasted the better part of this week, with brief blips of functionality," said one partner in an e-mail to CRN. "SMB online functionality for payment processing, payrolls and messaging is completely knocked out and Intuit seems to be struggling with getting things back online for more than a few hours at a time."
The service failures appeared to be on the same scale – or worse – as the multi-day outage that hit Intuit’s hosted SMB applications last June.
A posting late Friday from Kiran Patel, executive vice president and general manager for Intuit’s Small Business Group, apologized for the outage and said the company was working to correct the problems.
"This was a disappointing week, both for you and for Intuit," Patel said in the statement, posted at 8:53 p.m. PDT Friday. "And yes, that may be an understatement."
Patel blamed the Monday and Tuesday (March 22 and 23) service failures on human error during scheduled maintenance operations. Inuit changed its network configuration and inadvertently blocked customer access to a portion of the company’s servers. Those initial disruptions lasted only a few minutes while the change was undone, but restoring full service on both days "took longer than we would have liked," Patel said.
A surge in traffic overloaded the servers when connectivity was restored, so the company opted to restore service "at a purposely measured pace to closely monitor the system and prevent another overload," Patel said.
On Friday the same services went down. While Patel said they were restored "much more quickly," the company was still trying as of late Friday night to determine what caused that failure.
"In all cases, we know there was no security breach or attack on our servers," Patel said. "And we know that your data was not lost or damaged."
Patel acknowledged that for Intuit customers the service failure "goes beyond disappointment and beyond frustration."
"Let's start with an apology. Quite simply, this shouldn't happen and there's no excuse for this performance. We didn't meet our own high standards for dependability and customer service," Patel wrote. He went on to say that Intuit "wanted to do right" by customers whose businesses were affected by the service disruptions and provided 800 numbers for them to call.
Patel noted that "explanations and apologies don't mean much if we don't fix the underlying problems." Intuit is working to build resiliency and redundancy in its new products and data centers "so we can keep your services up and running in the event of a failure with minimal downtime," Patel said. Intuit is also improving the way it operates its data centers to reduce the risk of human error, he said, without providing additional details.
While an increasing number of companies are relying on hosted or on-demand applications to run their business, a number of highly publicized service failures from such companies as EMC Atmos, Microsoft, NetSuite and Salesforce in recent years have given some pause before fully trusting cloud computing.