Five Business Hours After Tornado, Joplin VAR Back Online11:42 AM EST Thu. Jun. 02, 2011
John Motazedi is standing on a large pile of rubble, looking for anything recognizable or salvageable in the wood and concrete and God knows what else that used to be the headquarters of SNC Squared, the Joplin, Mo.-based solution provider business he's owned since 2007.
It's been five days since a Class EF5 tornado carved a one-mile-wide by five-mile-long path of destruction on May 21, destroying an estimated 30 percent of Joplin, an estimated 8,000 buildings, 300 businesses and at least 140 lives. Looking over the debris that was his office building, Motazedi points to where his desk used to be. Nearby is a commode and a package of unopened toilet paper fairly close to where the bathroom used to be. But the desk? Nowhere in sight (click here for video of Motazedi surveying the damage) .
|View the Slide Show|
The St. John's building has become an iconic symbol here for the destruction that Mother Nature can cause. What once served as a vital element in this close-knit community of 50,000 in Southwest Missouri is now just a lonely shell with blown-out windows eerily exposing its devastated innards.
"We could never see the hospital from here before. This area was so green and there were so many buildings," Motazedi recalls from SNC Squared's address. Now they're all gone. Many of those buildings housed doctor's offices and other health-care businesses, conveniently situated near the hospital. Many of those businesses were Motazedi's clients. They were part of the reason he picked this spot for SNC Squared's headquarters.
Of course, all that's changed now. The tornado, which occurred at 5:41 p.m. on Sunday, May 21, destroyed SNC's office, as well as everything else around it. Motazedi was still looking for a new office on May 31, as were some of his physician customers. Almost one-third of SNC Squared's 75 health-care customers were impacted by the tornado.
"We liked to be near our clients. The problem is everyone was around the hospitals. Now there is no [St. John's] hospital," Motazedi said.
One customer closed his shop to take employment at another facility, and a second customer is leaving town for good. Four of his five largest customers sustained tornado damage, and about 460 of the 970 workstations that he manages as an MSP went down, he said.
Yet, incredibly, Motazedi was back online himself by 2 p.m. Monday. After picking up the shambles of his own business first, he turned his attention to the clients that needed him most. He had 100 percent of his clients' networks back online by 5 p.m. on Thursday, May 25, thanks to some quick action, flexible suppliers, patient customers and a lot of help from some solution provider peers both near and far. Here's how he did it:
Next: Disaster Strikes
The tornado sirens were sounding and the weather seemed to worsen. Quickly. It was Sunday night, just after 5 p.m. and Motazedi gathered his wife and three kids and headed to the family's basement in a wooded residential neighborhood of Joplin. The family stayed in the basement about 30 minutes, during which time they lost power but sustained almost no damage to their home.
But Motazedi was worried about SNC Squared's office because he knew that the magnetic doors would only remain locked for another hour or two on battery power if the lights were out in that part of town a couple miles away.
Motazedi grabbed the keys to the building and with his family in tow started to drive to the office. They had not gone far before realizing that a tornado had indeed struck Joplin and decimated whole neighborhoods.
"We just saw this massive chaos. I'm trying to weave my way through it. The kids are screaming and crying. My wife is saying, 'I can't believe this. I can't believe this,' " Motazedi said.
The Motazedis spotted a couple with four small children, including an infant covered by wet towels, walking in the general direction of a hospital. Motazedi pulled over and packed the family, who were in a daze, freezing cold and suffering from multiple lacerations, into his car and drove them to Freeman Health System Hospital nearby.
At Freeman, Motazedi saw a familiar pediatrician and asked if she needed help. The doctor said she was shuttling supplies to Freeman from St. John's, which had been heavily damaged by the tornado. Motazedi left his family at Freeman and got back in his car, following the pediatrician back to St. John's.
"The building was completely destroyed. It literally looked like a bomb went off. But we climbed through the debris and got gloves, peroxide, stitches, Band-Aids, anything we could dig out," Motazedi said. He made multiple trips back and forth before St. John's security said the building was not structurally sound and shut down access.
Back at Freeman, Motazedi gathered his family and headed home to put the kids to bed. He had gotten a text from one of his techs that the office was a wreck. About 11:30 p.m., he found a tarp and drove to the office to check out the damage. Or he tried to, at least. Debris and downed power lines in the road prohibited vehicle access, so Motazedi had to hoof it in the dark. He thought he was in the right spot, but the total devastation made finding a landmark impossible, he said.
"I got to the street and said, 'I wonder where my building is.' I turned around and saw the L-shaped parking lot and realized that this was actually my parking lot. I looked over [where the building should be] and I realized, 'That's my business.' It was kind of like, 'Wow, there's nothing left. I hope our backups are good,' " Motazedi said. "[There] was actually a sense of relief. I knew we didn't have any techs working over here. I had immediately texted all my techs and they all had responded that they were fine. So no one we knew was injured [nor] their immediate family. That was a relief. All this stuff can be replaced. This is insignificant."
Next: Salvage Mission
In the dark, Motazedi found his server rack, which had "accordioned" down with the weight of the fallen building. He put a tarp over the server rack because up to two more inches of rain were expected later that night.
Meanwhile, a nearby search and rescue team saw Motazedi, who had a flashlight, and summoned him over. They said there might still be bodies in some buildings to the south. "Luckily, we didn't find anything," Motazedi said. "I went home about 1:30 but didn't sleep at all. At 5 a.m. I just got up."
He went back to the remnants of SNC Squared's office and started to pull out any equipment and supplies that could be salvaged.
"When I got there I saw the [server] rack; a wall had fallen on it but it seemed dry because of the tarp. We needed Sawzalls and wire cutters to get the stuff out. We poured water out of a couple of them. We got them to my house and opened them up, blew them out and dried them off. I found a network switch, said some Hail Marys and turned them on," Motazedi said.
Almost miraculously, many of the servers fired up, although some firewalls and switches had to go in the trash can. He made a list of equipment he still needed to create a temporary NOC in his basement and had employees hit every store within a 40-mile radius to retrieve the items.
"We had some spare firewalls at my house. We did the routing. We worked with three different companies: Zenith [Infotech], Datto and eFolder. All of them were phenomenal," said Motazedi. "We had to get power from every outlet in the house. We have extension cords coming in through windows, from the garage, from upstairs to power all this stuff. We actually had 21 servers at one point running [in the basement]."
At 2 p.m. Monday, clients were back up on all three solutions, he said, only five business hours after the tornado struck the night before.
If the servers had failed, Motazedi said he could have had replacements within 24 hours, but he was proud of the fact that he had customers online the first business day.
SNC Squared had a "pseudo [disaster recovery] plan," Motazedi said. "It came out of a plan for what happens if an employee goes rogue. What key things do you need to protect? That included passwords and access to servers. Then, do you have all the vendor information you need and so you need to secure any portals?"
Motazedi credits help from other VARs in his Heartland Tech Groups chapter, including at least two who traveled from Minnesota and Arkansas to provide on-the-ground support. Executives from another Joplin VAR, Heartland Technology Solutions, as well as Network Data Services in Little Rock, Ark., and Success Computing in Minneapolis joined SNC employees on salvage missions from customer locations.
"We were hosting clients' servers in our NOC at 2 p.m. Monday. The remainder of the day, we started contacting every one of our clients to say, 'We're showing you down in LabTech and wanted to confirm that you are down.' We put together a list of our managed services clients and put them in priority followed by our break/fix clients," Motazedi said.
Next: Back In Business
Teams went to each location to salvage equipment and take inventory. They brought whatever equipment they could find to Motazedi's basement, now turned into a NOC. Techs also contacted Zenith Infotech, eFolder and Datto to get the last image backed up to the cloud.
"Luckily, most of them didn't see patients that Sunday so they all had valid data good as of Friday night or Saturday morning," Motazedi said. "The problem we were running into is they needed the patient information. Their server was up, but they had no way to access it. We had clients coming to my basement, logging onto their server. We provided printers and scanners so they could print out reports and their schedule of patients."
Meanwhile, SNC Squared was ordering laptops, UPSes and other equipment, knowing that customers were going to need new products to resume business.
Craig Hillyard, CEO of Network Data Services, one of Motazedi's peers from Heartland Tech Groups, arrived Monday to help with that task. "We came to the decision that if we don't order it now, when are we going to get it in? It took us three days to get the equipment in so now when people have [new] offices we've got the ability to get them back up and put equipment in their office today, whereas a lot of other people are just saying, 'Well, what do we need?' " Hillyard said. "John's way ahead of the ballgame and he's actually got doctors operational and [they] have the ability to see patients and access the systems. We're on Day 5 of a major tornado wiping out an area. His business is functional and his clients are up and running. It's phenomenal."
Motazedi is understandably proud of the fact his business was back only five business hours after his office was demolished. He cautioned other VARs to make sure they would be able to do the same if disaster strikes. Before you can help your clients, he said, the first step is to make sure your own business is back first.
"Can you come up in five hours? If you can't or don't have a plan, you can't help your customers. There's no way. If you've got a house in shambles, chaos in your business or your business doesn't even exist, you can't help anybody," Motazedi said. "You need a good strategy in place in terms of what's going to actually happen, how are you going to roll it out and then practice it."
Coincidentally, SNC Squared had done a test run a week before the tornado hit, shutting down servers to see if it could boot up from its backups.
"We came up just fine [but] that's stuff everybody puts by the wayside," he said. "And don't be afraid to ask for help. I posted something on Facebook that said, 'This is my new office', and it was a picture of my 'new office.' I got up Sunday morning not knowing that by Sunday night I would go through a major disaster. I wasn't planning on it. It was just a beautiful day."