McAfee: Partners Were Crucial In AV Glitch Cleanup
In April, when a faulty antivirus signature update took down thousands of McAfee customers' PCs, the company saw an opportunity to make lemonade from lemons -- with the help of its channel partners.
By enlisting channel partners to help clean up the mess, McAfee took a unique approach to a common problem, one that nearly every security vendor has faced at one point or another.
How'd that work out? McAfee says partners were instrumental in speeding response times and mitigating the impact of an issue that affected approximately 1,600 enterprise customers and 11,000 consumer customers. As a result, McAfee was able to resolve the situation without losing any customers, a scenario that seemed inevitable when the glitch first reared its head.
"We had people out at customer sites almost immediately, and we did not lose a single renewal in Q2 from customers that were affected by the issue," Alex Thurber, senior vice president of worldwide channel operations at McAfee, told CRN this week. "We had a number of customers impacted in a major way, but every one of them renewed."
Thurber says McAfee has figured out exactly what went wrong with the botched antivirus update and has policies and procedures in place to ensure it doesn't happen again. But if it does, McAfee wouldn't hesitate to recruit partners to help, he added.
"If there is such a problem in the future, we will absolutely engage partners from the beginning," said Thurber. "We do think this could come to be seen as a best practice for the industry."
The faulty antivirus signature, which McAfee issued April 21, erroneously flagged Windows XP Service Pack 3 as malware and caused PCs to display the dreaded "Blue Screen Of Death" and then enter an endless reboot cycle.
McAfee partners learned of the problem after a deluge of calls from panicked customers. "Everybody that got that thought they had a virus. It brought a lot of our customers to their knees," said one solution provider, who asked not to be named.
Although McAfee initially downplayed the impact, furious customer feedback caused it to quickly change its tune. McAfee devised a plan to send partners out to affected customers' sites to provide remediation services. Partners would also provide up-front health-check and consulting services, with McAfee reimbursing them for their travel expenses and time.
"We refused to come back with the excuse that these things happen, and we made the overall experience a positive one," Thurber said.
Next: How Many Partners Actually Helped Out?
Interestingly, McAfee never publicly announced the program, but Thurber says "more than 25" partners took part in a variety of roles. He declined to offer a more precise figure, explaining that "wherever there was a partner engaged with an affected customer, we worked very closely with that partner to resolve the problem."
Doug Hollenshead, president and CEO of Future Com, a Bedford, Texas-based security solution provider, had several of his large customers affected by the issue but says the cleanup went smoothly. "We went one-by-one through our affected customers and got them updated and back on line. Within couple weeks it was behind them," he said.
Hollenshead believes McAfee helped limit the damage by reaching out to channel partners at an early stage. "This was a great way to go about it because it could have potentially been a far reaching problem that affected their entire customer base," he said.
James Foster, CEO of Ciphent, a Hanover, Md.-based solution provider, says the health-check and consulting services gave VARs a chance to offer customers additional value beyond just the remediation. McAfee's response is something he'd like to other vendors adopt in these sorts of emergency situations.
"The systemic issues were mitigated quickly and we didn’t have any customers that were affected more than a few days," said Foster. "It was nice to have a vendor call all hands on deck and reach out to partners."
Has McAfee provided a roadmap for other security vendors that find themselves in these situations? It's tough to say, since such a tiny percentage of McAfee partners were actually involved in the cleanup. Security solution providers contacted by CRN couldn't remember any previous cases where a vendor has turned to the channel for help in remediating such a problem.
In any event, it's hard to argue that fast response in these situations isn't a good thing, and any security vendor that can marshal its channel toward the resolution of a widespread problem would be wise to take advantage of this capability.