Solution Providers, Vendors Help In Disaster Recovery

By Joseph F. Kovar, CRN 10:41 AM EST Mon. Sep. 17, 2001

Solution providers and vendors are working to help get clients' infrastructures back in operation as soon as possible after the crash of two hijacked jetliners into the Twin Towers of New York's World Trade Center.

Whether their entire data center disappeared in the resulting ball of fire, or miles-long police lines are stopping employees from grabbing their laptops from otherwise undamaged buildings, companies with facilities in the city's financial district face the possibility of losing access to their IT infrastructures.

Solution providers agreed that most companies involved in the disaster had plans in place for either disaster recovery, which focuses on getting a company's IT infrastructure back online after a disaster, or business continuity, which aims to ensure that all the details that keep a company running, such as how to contact employees, are covered.

Even so, "there's no way for anyone to prepare for that kind of disaster," said Al Zilinsky, senior data management consultant at Champion Solutions Group, a Boca Raton, Fla.-based solution provider.

Twenty-two clients of SunGard Business Continuity and Internet Services, Wayne, Pa., declared disasters, said Dave Palermo, vice president of marketing. SunGard is one of the largest providers of disaster recovery services.

By "declaring," as Palermo put it, such clients invoked their contract with SunGard to put in place the specified platform and prepare to help them resume operations.

SunGard has 26 disaster recovery facilities across the nation, as well as several mobile data centers based in 18-wheel trucks. Palermo said the clients in the New York area were routed to one of six facilities on the East coast.

How fast the clients came up depends on how they archived applications and data available, said Palermo. At the low end of the spectrum, which SunGard calls "recovery," getting online depends on how quickly clients can get the applications and data to SunGard. At the high end, or "failover outsourcing," clients have high-availability solutions with automatic failover and data replication.

"That's why we advocate high-availability solutions and replication," he said. "It's better than tape. Tape can take 24 hours to load. High availability costs more. But the costs have dropped substantially over the last couple of years, thanks to equipment and T3 line costs."

Clients who backed up or replicated their data to facilities owned by StorageNetworks, the Waltham, Mass.-based SSP, had immediate access to that data, said Lou Berger, senior vice president of technical qualification and certification.

Other clients who used StorageNetworks to manage their IT infrastructures had immediate access to the management information and are working with StorageNetworks to build out alternative data centers, Berger said.

StorageNetworks had about a dozen clients in the areas surrounding the building collapse, said Berger. He declined to name the clients, but the company has previously listed Merrill Lynch, which had a large operation in the World Trade Center, as a client.

Disaster recovery is insurance, Berger said. "Every customer gauges the amount of insurance they want to pay for," he said. "If they replicated data, they are already up and running. If they did backups, it might take longer."

Jeff Boston, senior vice president of consulting at Forsythe Solutions Group, a Skokie, Ill.-based solution provider, said he is still trying to confirm the actual number of clients who were affected by the attack, but that all their disaster recovery plans worked.

"Many didn't have their key servers or infrastructures in the buildings," Boston said. "But they had PCs, laptops, and other devices accessing those key servers."

While the immediate IT focus in disaster recovery is making sure key systems are up and running as soon as possible, Boston said that over the long run the bigger exposure for clients in New York is the ability to access data not on mission-critical infrastructures.

"What's different from the Oklahoma bombing and other disasters is that in those cases, people eventually get physical access to the buildings," he said. "They get access to physical data and departmental computing. . . . It will take several days to come to a full realization of what happened to their department computing. Plus, everyone has briefcases, PDAs and laptops. These will be lost."

In order to help end users rebuild their infrastructures as quickly, several vendors have designated AMC, a solution provider with warehouse facilities and its own delivery trucks, as a crisis center provider, said Mark D. Romanowski, senior vice president of client services and business development.

While Romanowski would not disclose the names of the vendors, he said they chose AMC because of its proximity to the blast area and the fact that the solution provider can quickly put a couple hundred people, including numerous high-end engineers, to work. "We are one of the largest, if not the largest, in New York," he said. "We have more technical professionals than anyone."

AMC is working with the manufacturers' clients to help them get their storage and related infrastructures up and running as fast as possible, either on the customers' sites or in the AMC warehouse. The solution provider, however, does not provide disaster recovery services like SunGard, nor does it offer co-location services, Romanowski said.

Other vendors are also mobilizing resources to aid clients in the New York city area.

EMC, for instance, has opened its Internet hosting facilities as a temporary disaster recovery site to customers whose primary data centers went offline in the attacks, and are now running their businesses on their secondary facilities.

The vendor is offering the service free of charge out of its Hopkinton, Mass.-based Internet hosting facilities, currently used mainly for testing hosted storage services but also for offering hosted services to a limited number of clients.

Veritas Software is mobilizing a large number of "SWAT" teams consisting of people from the company's support, data recovery and engineering groups, as well as a sales engineer familiar with a particular client's account, said Gary Bloom, president and CEO. "There has been massive amounts of data loss, more than anyone can comprehend," he said.

Fujitsu Softek is providing products, services and support at no cost to clients struggling to bring their operations back online, a company spokesperson said.

The aftershock of the attack was felt in data centers outside the New York and Washington areas. "The Prudential building in Boston was evacuated," said Zilinsky. "This shows you need to have a lights-out operation, or be able to switch operations to another location."