Staples Cleans Up Its Apps

It's every online shopper's frustration and what retailers fear: A customer logs on to do some shopping conveniently from work or home, navigates through the site, adds goodies to a shopping cart, proceeds to the checkout and, bam, the connection is lost or the site freezes up, and the customer shuts down, never to return again.

Sound familiar? Let's hope that isn't happening to your customers. That's especially true given how fast online sales are expected to grow in the future, assuming, of course, customers can actually complete their transactions. According to a recent report from Forrester Research and Shop.org, online sales are expected to reach $144 billion in 2004, up from $114 billion in 2003.

With billions of dollars in potential sales at stake, many retailers are taking great pains to ensure their sites stay up and running, regardless of traffic spikes or holiday rushes. Retail giant Staples, for example, has taken this to heart for both customer-facing and corporate applications, enlisting the help of Star Quality, a 6-year-old Hopkinton, Mass.-based consulting firm that focuses on automated testing and software-quality assurance.

"Automated testing is one of the components of our software development life cycle that helps decrease our costs and time to market," says Kathy Murray, senior manager of quality management at Staples.

Previously, Staples had been doing the majority of its testing in-house. "Staples is dedicated to launching programs that make it easy for our customers to buy office supplies," Murray says.

The problem was, the internal testing wasn't as smooth or as efficient as what Staples needed, so it looked to outside experts for help. Staples' senior management was familiar with Star Quality from previous business dealings, Murray says, and was ultimately selected "because their products and methodology supported Staples' goal to make it easy for our associates by providing a proven automation framework and a library of reusable code," she says.

More specifically, Star Quality was called in to develop and deploy an automated testing and quality-assurance strategy for Staples' consumer Web site and its StaplesLink site for business customers, as well as testing for its IBM iSeries, point-of-sale (POS) and corporate applications.

The project included the implementation of Segue Software's SilkTest, an automated tool for testing the functionality of enterprise applications in any environment. SilkTest enables users to verify application reliability in short testing cycles. Star Quality also deployed Segue's SilkPerformer, an automated load and performance testing system for maximizing the performance, scalability and reliability of enterprise applications.

Using SilkPerformer, Staples is now able to predict breaking points in applications and the infrastructure before being deployed. SilkPerformer simulates thousands of simultaneous users working with multiple computing environments and interacting with various application environments.

Star Quality built a customized layer on top of the Segue software based on its StarTest platform to improve the ease of use and maintenance of testing scripts.

"We've customized and married the two products in order to lower the bar in terms of the skill level to be successful with test automation," says Brian LeSuer, partner at Star Quality. In addition, Segue's SilkTest Blue Express add-on for testing IBM iSeries applications was used.

"We piloted the solution on an e-commerce application release and, based on the success of that pilot, implemented an internal training program and rolled the solution out to our other e-commerce applications," Murray explains.

More specifically, Staples is utilizing the Segue/Star Quality software for both performance and functionality, LeSuer says. For example, the Staples.com Web site is tested for performance when stressed with many users at once. "We can simulate hundreds [to] thousands of people using the Web site at once and see how it holds up," he says. On the functionality side, he says, Staples tests POS or cash-register functionality in a Staples lab environment to determine if an application does what it is supposed to do.

One of the biggest challenges, LeSuer says, is that when testing a Web store like Staples.com, everything is constantly changing. "You can't rely on prices staying the same or SKUs being in stock," he explains. "There are some interesting technical challenges with building automated tests that can be run over periods of time. So we built tests that were smart, that could figure out run-time [and] what the expected results should be based on the data that day."

He adds that companies like Staples use these solutions on a daily basis, with new projects and applications being introduced all the time. "Staples is always working on the next version of every application," LeSuer says. "It's pretty much an around-the-clock kind of thing."

Murray agrees, saying Star Quality had to tailor its solution to address the needs of Staples' unique site content. "They worked closely with us and Segue Software in order to successfully address challenges," she says. "Staples, Star Quality and Segue were all committed to the success of this partnership."

One reason the relationship between the two companies works, LeSuer explains, is that Star Quality serves as both a business adviser and a technology provider. LeSuer, for example, teaches classes on quality-assurance processes and test planning at Staples University, the retail giant's internal education center.

With an IT department of 600 in North America, Murray says that Staples often looks to outside solution providers for project help. "We look to third-party technology providers to help us meet our business needs, drive profitable sales, improve our margins and get the most from our assets," she says.

When working with third parties, Murray says that communication, expertise and partnership are the three most important aspects she looks for. In Star Quality's case, daily communication was critical to the implementation's success.

In general, LeSuer says it's tough to gain the respect of the companies consultants work with. Star Quality got in the door because Murray was familiar with the company and the Segue products. But, he says, "We had to prove ourselves, and that was made clear to us from the start."

The relationship has hit three years now, and Star Quality has proved itself and become a trusted adviser. "Staples relies on us to advise them as far as where they should be going next, what tools they need, processes they need to implement, and we help them," LeSuer says.

LeSuer also points out the importance of seasonal selling for online retailers and getting the work done in time. "You have to get stuff done in time for Christmas, tax time, back-to-school, or the initiative doesn't happen at all," he explains. "You miss that opportunity and you don't get it back again."

Preparing For The Rush

With the holiday season upon us, many e-tailers and retailers are holding their breath and waiting to see if months of Web-site preparation and testing will pay off. Forrester estimates that online holiday sales will grow 20 percent over last year and account for billions of dollars in sales, a large chunk of change for e-tailers.

Not leaving any losses to chance, e-tailers including BlueFly.com, an online designer outlet store, are relying on a variety of third-party tools to help ensure their sites withstand the crushing load and demands of holiday shoppers. BlueFly.com, for example, is using the same testing tools from Segue Software that Staples and others are using in preparation for this shopping season. BlueFly.com enlisted the help of Testware Associates, a Somerville, N.J., reseller of testing software solutions, to implement Segue's SilkPerformer and SilkTest software.

Other retailers depending on Segue software to ensure their sites are ready for the holidays include Avon Products, Gap, Costco and Sears.