January 31, 2006, 12:00 AM

Beyond the Numbers

(Page 2 of 3)

“Allowing organizational silos to build up means that no single department really knows how all the performance and customer activity data gathered fits together,” explains Daniel Heimlich, vice president of marketing for Netuitive Inc., a Reston, Va.-based provider of performance monitoring software. “The data just sits in a vacuum and identifying the cause of the problem is like trying to find a needle in a haystack.”

Putting performance problems into context requires retailers to notify their performance monitoring services to focus on specific metrics and to produce reports on isolated incidents. “Once retailers educate their service provider about which performance metrics they care about, isolated incidents can be spotted and put into perspective against other isolated incidents that at first glance may not seem connected, but are actually creating a larger problem,” explains Alistair Croll, vice president of product management and co-founder of Coradiant Inc., a Poway, Calif.-based performance monitoring service. “The more angles from which a retailer can attack a performance problem, the better the chances of quickly determining its cause and impact on their business.”

Prioritizing the order in which performance problems need to be addressed starts with setting clear objectives. At the start of each quarter, Priceline.com gathers all departments to set marketing, merchandising, and performance issues for the next three to six months, and determine how best to achieve those goals. “Everyone knows what the goals are, then we figure out as a team the best path to march to in order to achieve them,” says Priceline.com’s Rose. “We have silos within the company, but communications between departments is not an issue.”

Measuring the suppliers

As an e-commerce portal that relies on third-party suppliers, such as airlines and hotel chains, to provide information, as well as links to their sites, Priceline.com measures performance data on its suppliers, as well as data on specific customer sessions. The data helps Priceline.com determine whether a problem resides within its infrastructure or that of a supplier. In the event a performance problem resides within a supplier’s site, Priceline.com will contact the supplier and provide details of the problem.

Priceline.com, which uses Keynote Systems Inc. and Netuitive to measure site performance, measures about 30,000 metrics on its site, including information requests of its database, the frequency of error messages, and CPU utilization throughout the day.

It is this kind of vigilance that keeps Priceline.com from tripping over itself when it comes to interpreting the significance of performance data. “The better a site monitors infrastructure, the better the chance of keeping a customer by not losing them to a performance problem,” says Rose.

Netuitive aids Priceline.com in this endeavor by capturing actual customer sessions where a performance problem has occurred, rather than relying on recreating the session. Replaying an actual session, which is time stamped, makes it possible to see the problem that occurred as the customer saw it, thereby giving the retailer a better sense of where the problem originated and its severity. At the same time, it resolves discrepancies between how IT and other departments view the problem.

Warning signs vs. trouble

“Some performance indicators are just warning signs as opposed to actual trouble,” adds Rose. “That’s why our performance metrics are prioritized and we are looking at actual sessions now. It’s probably something we don’t do enough of.”

One trap retailers want to avoid when prioritizing performance problems is relying strictly on customer reports of problems. By the time enough customers complain so the retailer can spot a trend, the problem may have become so full blown it impacts the credibility of the retailer’s brands in other sales channels.

“Customers aren’t always going to tell you why they abandon their shopping cart of if they were dissatisfied with the shopping experience,” stresses Crutchfield’s Weiskircher. “Besides, does any retailer really want to learn about a problem from a customer? If a customer brings a problem to the attention of the store manager, the manager might wonder what the staff is doing other than their job of servicing the customer. Customer feedback is important, but it does not give the whole picture.”

For retailers intent on including customer feedback when prioritizing web site metrics, a proactive approach is considered the best. Asking the customer to fill out a performance survey at the conclusion of a session via e-mail is one strategy. “This can help retailers understand where their site is weak and start to prioritize their investment in site performance,” says Bonny Brown, director of research for Keynote.

Some site performance experts counter that most retailers already know where their sites are likely to fall down, as they know the amount of resources put into building the site. “Retailers know what part of their sites were built haphazardly, and if they don’t, they are going to be in for some big surprises,” contends Nielsen/NetRatings’ Dougherty. “Retailers who rely too much on customer feedback to spot a problem end up following the problem, not being out in front of it.”

Still, customer feedback ought not to be completely dismissed. “Customer feedback can provide deeper insight to the impact of a performance problem,” says Brown, who acknowledges there is much debate over the weight to be given to customer feedback regarding performance.

Regardless of the data used to measure site performance, one aspect retailers and site performance experts agree upon is that e-commerce has matured to the point where retailers must adequately budget for performance monitoring and set clear performance goals across the enterprise. Scrimping on performance measuring is only going to create an environment that allows small problems to go unchecked and become full blown. Ultimately, such problems negatively impact sales and the retailer’s brand.

“The days of brainstorming about how to fix a problem are over,” says Brown. “The time has come for retailers to get more scientific about performance monitoring and have a clear idea of which metrics are most important to profitably running their business.” l

Peter Lucas is a Highland Park, Ill.-based freelance business writer.

Picking up the pace of performance metrics

Once a retailer has aligned internal processing goals with performance metrics, the final piece of the puzzle is determining the frequency with which the measurement is taken.

comments powered by Disqus




From The IR Blog


Terri Mock / E-Commerce

How online jewelers fared this Valentine’s Day

The key takeaway: Start early, because sales tail off in the last few days before ...


Cynthia Price / E-Commerce

4 tips for improving email marketing results

Every piece of data you collect can help you serve your audience exactly what they ...