How To Earn Maximum Trust From Your Managed Services Clients

Imagine, for a moment: One morning, a technician wakes up, goes to work, hatches a plan. He works for a managed service provider (MSP) that develops and manages grocery store inventory-control systems, remotely maintaining and updating those systems for clients spread across several time zones. He recently learned that his company is changing the employee bonus structure and he's unhappy with the result, to put it mildly. His response? Retaliatory and unconventional; he inserts a particularly malicious bit of code into the inventory-control software developed by his employer and installed on all of its clients' networks.

So, one morning, employees of two different East Coast grocery-store chains go to work and start up their inventory-control software, only to watch those systems first crash, then refuse to reboot. The MSP responds very quickly, moving to head off the infection before the morning shift starts at three more chains farther West. Where the damage has already been done, however, options are limited; the MSP is forced to fly staff out to rebuild the systems at each of the affected client sites. All in all, those systems are offline for two days.

Unfortunately, this isn't a fairy tale, or even hypothetical, according to Dawn Cappelli of the Computer Emergency Response Team (CERT) at Carnegie Mellon University's Software Engineering Institute. Though barred by CERT's confidentiality rules from disclosing names and other identifying details, Cappelli describes this true story drawn from CERT's files as "an illustrative case."

The very nature of the services MSPs provide demands a qualitatively different type of trust relationship with their clients than other solution providers typically maintain. When a business hires an MSP, it opens up its own security perimeter to allow that MSP in. The client isn't just betting on the MSP's competence and capacity to carry out specified tasks. The two companies are entering into an ongoing relationship, in which the client places a great deal of faith in the MSP's leadership, employees, partners and technical capacity to defend itself from attack. As Doug Howard, COO of Mountain View, Calif.-based managed security services provider Counterpane, puts it, "We're the guys watching your network, so who's watching us?"

In a very real sense, the client is putting its head into the lion's mouth; as an MSP, it's up to you to make sure they feel justifiably safe in doing so.

There have been no recent publicized client security breaches that are traceable to an MSP, and neither MSPs nor their clients are eager to speak up about unpublicized ones. Indeed, almost no one in the industry really wants to talk about the security and trust issues inherent in managed services, and understandably so. Given the public's inclination to react to perceived IT security issues out of fear, uncertainty and doubt rather than informed analysis, many are rightfully concerned that MSPs as a whole might be tarred--unfairly--as an inherent security risk.

Several recent incidents have highlighted the trust issue, however, and its increasing significance in the growing MSP market. On Oct. 31, 2006, federal agents arrested the CEO of White Plains, N.Y.-based MSP Compulinx on charges stemming from an alleged scheme to use personal client and employee information in fraudulent credit applications at a number of banks between 2003 and 2006. On March 30, the Chico, Calif.-based MSPAlliance held an emergency meeting to assess its membership approval and accreditation processes after discovering that two members and one applicant may have engaged in questionable business practices; the applicant in question had, among other things, an employee awaiting trial for allegedly embezzling more than $100,000 from a previous employer. The MSPAlliance also reports receiving hundreds of applications from "MSPs" in Nigeria, the Middle East and Russia that lack verifiable credentials.

In the long run, silence on this front could prove an extremely costly strategy both for individual MSPs and the industry as a whole. Sooner or later, a high-profile public incident is inevitable. When that time comes, if MSPs haven't already educated the public and built strong trust relationships with their clients by engaging with them on this issue, the reaction is likely to be swift and harsh. In 2005, for example, three employees at an India-based outsourced call center allegedly stole more than $350,000 from four Citibank customers. The resulting crisis of trust in the entire Indian business-process outsourcing industry threatened to cost it as much as 30 percent of its growth, according to a Forrester report, although the loss was largely mitigated by a nationwide scramble to improve security and shore up confidence.

NEXT: Why your reputation is on the line.

While shifts in the market and the sophistication of the client base may call for some changes, building and managing trust relationships is integral to the experienced MSP's business model.

"Trust and integrity aren't just an aspect of how we do business," notes Tracy Butler, president of the Wood River, Ill.-based MSP Acropolis Technology Group. "For any MSP, they are a specific part of the product you sell, although people may not realize it or think of it that way."

For solution providers venturing into the MSP space, however, this trust factor may require some significant adjustments. "In the past, if you did a project for a client, installed equipment and walked away, that's something the client would take a risk on and that would be OK," says Ian James, president of Pittsburgh-based MSP Red Square Systems. "Now, they're betting the farm on you, so your reputation matters, and trust becomes the dominant issue. How do you establish in the mind of a client that you're a trustworthy organization?"

Justin Crotty, Ingram Micro's vice president of services, agrees. "Everyone has to differentiate themselves--VARs, MSPs, whatever. The need to differentiate doesn't change if they become an MSP, but the way in which they differentiate certainly will. For those guys, it's going to be about trust."

If you're going to earn a client's trust, you have to start by actually being trustworthy. It should go without saying that this includes refraining from lying, cheating or stealing. In the MSP context, it must also as an absolute minimum include ensuring that the MSP doesn't increase the client's risk exposure. In one sense, this should be a fairly straightforward exercise for a conscientious IT professional: following security best practices, maintaining careful HR policies, conducting exacting due diligence when selecting partners. It also, however, demands awareness of clients' levels of risk tolerance when making decisions that might seem internal to the MSP. Any risk the MSP assumes, it assumes for all of its clients.

From a business perspective, being trustworthy is necessary but not sufficient; it doesn't help your client relationships much if they don't recognize how trustworthy you are. As such, MSPs must be able to communicate openly and effectively on these issues with their current clients and differentiate themselves from their less trustworthy competitors to new clients.

Communicating about trust, unfortunately, can be anything but simple. Most clients turn to MSPs precisely because they lack the very technical expertise they need to know whether or not an MSP is feeding them a line. "It's hard for an end customer to do an evaluation on a singular baseline standard with which to compare everybody," Crotty says. "They have to do due diligence, but regardless of what an MSP tells them, it's hard for the customer to know what's good practice and what isn't."

One approach is to rely on a trusted third party to give trustworthy MSPs some sort of seal of approval. Without some sort of third party to look to, explains MSP Alliance president Charles Weaver, clients will have no way to distinguish between honest MSPs and those that merely talk a good game. "When you and I go into a doctor's office or a hospital," he notes, "we assume that they're there because they've been vetted. They are held to a standard. The MSP industry doesn't have that, except for this accreditation standard."

For the moment, however, there's no accepted industry standard for MSP certification or accreditation. There are a host of certifications that look at skills, capacity and practice--ISO 17799, BS 7799-2 and -3, SAS 70, ITIL, eTOM--but none of these look at whether an MSP is legitimate or how it actually conducts its business on a daily basis. "Yeah, OK, I went through my SAS 70 type 2. Who cares?" Counterpane's Howard says. "In reality, if you knew what an SAS 70 type 2 is, it's a complete waste of time from an average customer perspective. All of these certifications are checklists that gain me some credibility, but that's something anybody with $50,000 and a contact at an auditor can get."

NEXT: Three basic tests an effective accreditation system must pass.

The need for some sort of accreditation is widely acknowledged, but an effective accreditation system will have to meet three basic tests: acceptance by the MSP community, recognition by the client base and a careful balance between cost and exactitude. If MSPs don't see the value in a program, it will simply be ignored, and if clients don't recognize the program, it provides little benefit to participants. The more detailed and comprehensive a program is, the more it costs to implement; the more expensive a program is, the more small MSPs are locked out. This particular balancing act is complicated by many larger and well-established MSPs' interest in raising barriers to entry and protecting their investments by blocking potential competitors.

Several groups have moved to fill the current accreditation gap, but none has yet met all three tests. The MSPAlliance has an active program intended to complement existing capacity- and skills-based certification by delving further into business and human-resources practices. According to Crotty, however, they haven't yet garnered widespread support within the MSP space. "I think organizations like MSPAlliance really aren't in the driver's seat in terms of their acceptance by the channel, as a channel standard or as a speaker for the channel."

Cisco is developing a certification program based on a combination of the ITIL and eTOM standards and Cisco's own specialization programs. The program is currently in the pilot phase, and Cisco expects to release it this year. As a vendor-based certification, however, it has little chance of becoming an industry standard. Moreover, Cisco is expressly targeting larger, well-funded MSPs.

CompTIA is also currently exploring implementation of its own "trustmark," but the program is in a relatively early stage and its precise outlines are unclear. "We view this as a critical need for our members," says CompTIA vice president of services Rich Rysiewicz. "We're working closely with our members to determine just what it should look like."

It Comes Down to Touch
Ultimately, however, the trust relationship between MSP and client is between the MSP and the client.

"I don't care how big the deal is or how many certifications you have," says David Dadian, CEO of Powersolution.com, an MSP in Ho-Ho-Kus, N.J. "Anyone can get certified. In the end, trust comes down to touch. You have to shake his hand and look him in the eye."

Counterpane's Howard agrees, for good or ill. "A lot of it, sadly, comes down to 'Did I, the client, walk out of that meeting feeling warm and fuzzy?' rather than 'Did I walk out of that meeting knowing that there's a clear differentiation?'"

The ability to keep a client feeling "warm and fuzzy" is difficult to quantify, but one place to start is with aggressive transparency. Whenever there's a question, always err on the side of disclosing more information and making sure the client understands the significance of it.

Sooner or later, every MSP will face a choice between painting themselves in the best possible light in order to make a sale and accurately disclosing risks or the limitations of their own capacity. Weaver believes that in such cases, disclosure and transparency should always come first. "If you're smart, you can present it in a really positive light, but your clients have a right to know if you're relying on someone else's NOC, or if you're not going to be available 24/7," he says.

The importance of reputation makes this good business as well as good ethics. Consider this: When something goes significantly wrong (and it eventually will) what will do more damage to your business? The deal you lost because of an excess of transparency, or the angry client who feels you betrayed their trust?