February 3, 2014, 11:38 AM

Test the limit

(Page 2 of 3)

For Things Remembered, whose multiplicity of options make simulating visitor patterns difficult, Compuware has proven a good choice, Lilien says.

When Things Remembered makes a change to its site, Lilien uses the vendor's self-service tools to run a load test that stresses the site to see whether the move will cause problems. "We have to know what our site can handle," he says. "Otherwise we're too late."

For example, the personalized gifts retailer last April gave shoppers the option to buy an engraved item online and have it waiting for them in a store four hours later. And while it tracked the percentage of consumers who selected the option during its busy seasons—Mother's Day, Father's Day and graduation—it remained uncertain how the site would respond during the hectic winter holiday rush.

Offering buy online, pickup in store in combination with personalization can tax the site, Lilien says. That's because the site has to recognize the items' idiosyncrasies, such as where they can be engraved, and check to determine which items are in stock at a particular location.

To avoid potential hiccups during the holiday rush, the retailer used Compuware's tool to run load tests about eight weeks before the start of the holiday season. In preparation for the tests Things Remembered forecast how many shoppers it expected on its site—it had an average of 990,000 monthly unique visitors in 2012, according to Top500Guide.com—and diagrammed about 30 different paths those shoppers were most likely to take, including those who would select the buy online, pickup in store option. The retailer then tested what would happen if more than 10 times more consumers visited the site than the retailer anticipated. It ran the test at 3 a.m. so that if the test traffic brought down the site few, if any, shoppers would be impacted.

The test helped the retailer find and fix weak spots, such as the site making unnecessary database calls. Those fixes were completed before November, helping the retailer speed up its site and avoid any significant downtime during the holiday season. Moreover, that performance likely contributed to the retailer setting sales records, Lilien says.

Knowing potential weak spots has helped The Jones Group Inc. create contingency plans in case any of its sites slow down, says Michael Hines, vice president of mobile strategy and e-commerce systems for the retailer, which operates several retail sites for brands including Jones New York, Anne Klein and Nine West. The retailer eight years ago began working with AlertSite because the vendor made it easy to set up the click path for both load testing and ongoing site monitoring.

Before last holiday season it used five of its most common click paths to run load tests that simulated the impact of traffic four times the number of typical visitors during the holiday season (which is roughly double its sites' non-holiday traffic). It discovered that the content on category pages on its sites like NineWest.com— multiple images, brief product descriptions, available colors and the price—was slowing the site down when it came under what Hines calls "extreme duress."

After identifying that content as a potential problem, Hines and his colleagues developed a plan to manually tweak the site's coding if it received an automated call from AlertSite notifying it of a site slowdown. The planned change was to present a fraction of the products on the category page. For example, rather than show a shopper all 91 pumps on NineWest.com, it would show her a smaller number, say 20 pairs, and require her to click through pages to see all of the items in the category.

"By making that small change, we'd avoid having an unusable site experience," Hines says. "Instead we'd be giving shoppers a reasonably good site experience." Even though the site didn't slow down to the extent that it needed to make the move during the holiday season, Hines says the planning was worthwhile because it was armed with a plan to avoid losing revenue.

AlertSite also helps The Jones Group provide an experience superior to its competitors, he says. The retailer works with AlertSite to regularly track load times for five of its competitors—both those on the same Demandware e-commerce platform that it uses, and those using other technology. Benchmarking against other sites, particularly those on the same platform, lets the retailer identify issues that it might not otherwise have uncovered. For instance, if the retailer finds that a product page on another site loads 700 milliseconds quicker than a comparable page on NineWest.com, it can dig into the features on its page to see if an element like multiple product views is slowing the page down. It can then adjust. For example, instead of loading alternate images before the site fully displays, it might load them after the main components on the page load.

"Over time if you don't pay attention, you can find that small additions to a page can make a site that used to be fast much slower," Hines says. "The only way to combat that is to periodically review our performance."

Online ticket marketplace StubHub, a unit of eBay Inc., doesn't want to wait to unearth any slowdown; it wants to know about any delay right away before shoppers have problems making purchases. That's why it works with three monitoring vendors—including Keynote and AlertSite—to closely track its site's performance, says Hanna Sicker, the company's site operations manager.

comments powered by Disqus




From The IR Blog


Anna Johansson / E-Commerce

Why is social proof big for niche brands?

A small online retailer that lacks brand recognition can get a big boost from high ...


Donn Davis / E-Commerce

Technology takeover: The fashion industry is next

We are now entering the third decade of the Amazon effect, and it is just ...

Research Guides