Webroot Update Crisis Gives MSPs A Chance To Step Up And Help Customers

MSPs said they are stepping up to the plate after a Webroot update yesterday inadvertently flagged Windows system files as malware and marked several major websites as phishing sites.

The Windows system files, marked as general malware, were then quarantined and left some applications unable to function. The error lasted 13 minutes, starting at 1:52 EST, according to a company forum on the issue.

The update also caused many false positives, most notably marking multiple popular websites, including Facebook, as phishing sites. Users were prevented from visiting those sites.

[Related: 2017 Security 100: 20 Coolest Endpoint Security Vendors]

id
unit-1659132512259
type
Sponsored post

In a letter to MSP registered admins Monday, Executive Vice President of Product and Strategy Mike Malloy said Webroot is working to resolve the issue. He said the company is conducting a "thorough technical review" to understand what happened.

"We recognize that we have not met the expectations of our customers, and are committed to resolving this complex issue as quickly as possible," Malloy said in the letter. "We apologize for the pain this has caused you and your customers. Webroot appreciates your business, and our entire team is dedicated to being your most trusted partner. We did not live up to that in this situation, but we are taking the actions to earn your trust going forward," he said.

In a company forum, Webroot said the company had fixed the false positive issue. It said customers should check endpoints are turned on and connected to the internet to receive updates to resolve the issue. It advised customers not to uninstall the product or delete files from quarantine, as files will then be unrecoverable.

In a statement to CRN, Malloy said the company has been working to keep partners up to date on the issue since it was discovered.

’Since this issue came to light, we’ve been in communication with our partners. We have an open and interactive online community of partners and customers who share ideas and collaborate on solutions. We started posting solutions in the community Monday afternoon.

"Since then, we’ve been reaching out to partners via the phone and email to help with any issues they’re still seeing in their customer base and offering our ongoing support. For example, yesterday, we shared updates with our partners to use with their clients and today we communicated the instructions for a standalone streamlined fix," Malloy said.

MSP partners reached by CRN said they were still dealing with the fallout from the update issue, with many voicing their concerns on Twitter.

Chris Johnson, director of compliance and security services at Fort Lauderdale, Fla.-based Wheelhouse IT, said his MSP business manages around 4,000 Webroot endpoints. He said around a dozen clients were impacted in a "serious" way.

"Yesterday was crazy," Johnson said. "It was definitely crazy."

However, Johnson said it was also an opportunity for his business to shine. He said help desk managers manually remediated the dozen or so clients affected, and the company did some preventative work on its remaining clients to prevent it from becoming a serious issue. He said Wheelhouse IT was able to be proactive with the issue by doing its own constant monitoring of deployed products.

"We prevented what could have been a pretty big issue," Johnson said. "I think this was we took a few bruises on this one, but there were no life-threatening wounds inflicted."

Johnson said he felt Webroot did a good job quickly communicating, taking responsibility and remediating the issue. He said it could happen to any vendor in the security space.

"We were quite honestly impressed," Johnson said. "This is a vendor that genuinely cares about their partners and is quick to respond."

Johnson said his business would do due diligence with all its vendors to make sure there is a disaster recovery plan for any problems that may arise in the future.

"I think the kicker is using this as a teaching moment moving forward," Johnson said. "We can't just assume or rely on when a bad thing happens that a vendor is just going to be able to push a button and remediate. We have to proactively solve the problem."