Doran Robinson previously worked for healthcare information technology vendor athenahealth.
Once there was a time when web site performance meant site availability and page download time. But as online shoppers and online retailers have become more sophisticated, so has the notion of web site performance. Availability and page download times are coming more and more to be viewed as equivalent to having the store’s doors open and the lights on. They’re only the entry points to a customer experience; they are not the experience itself.
Retailers need to focus more on individual users’ experiences every bit as much as they focus on monitoring uptime and server availability, argues a group of site performance monitoring companies. “Definitions of downtime usually miss one dimension completely,” says David Jilk, CEO of site performance monitoring company Xaffire Inc. “They focus only on when the whole site is down, not when individual users can’t use the site.”
Retailers’ attention needs to be on all the things that make up a user experience, these vendors say. “We monitor the real performance of web sites and record every visitor, what they do, how long the pages take to load,” says Mike Dickey, CEO of ClickCadence, creator of the BeatBox monitoring product.
A number of perspectives
Online retailers today are looking at site performance from a number of perspectives. At one level is the monitoring of performance so as to know the experience the shopper is having. At another level is anticipating what shoppers will want and making sure that the site is ready to deliver when the shopper is ready to receive. And that calls for changes in the way retailers approach the Internet. “You have to be savvy in the ways you measure performance,” says Bob Hammond, CTO of Mirror Image Internet, a network for the delivery of online content, applications and transactions. “A lot of people think if they have a big pipe to the Internet, they’re fine.”
Not necessarily so, Hammond says. He argues that retailers need a distributed network to house and deliver what the shoppers want remotely, not from a centralized server. “The real value proposition that we offer is the ability to offload graphics and middleware,” he says. “That in turn makes the site richer to attract a larger set of folks and it allows retailers to add navigational features that make the site more interactive although heavier to deliver.”
Xaffire is one of the newest entrants into the web site performance monitoring business. It has created web site monitoring and recording capability in a software diagnosis product called Xfire that records all user sessions, scans them for problems, then allows tech support to replay the session to identify the problems.
That solves a big problem, Jilk says: “You’re limited to only a few ways to resolve a software problem,” he says. In most computing environments, tech support can ask the user what the problem was and try to walk trough a re-creation and resolution. Because of the self-service nature of the web, that’s usually not possible with online customers. “A bug might happen only when a certain confluence of events takes place,” Jilk says. “The only way to see what happened is to record the transaction.”
That, in turn, gives tech support insight into the session that’s not otherwise available, Jilk says. “Support personnel can see what the user saw and what was happening at the time the error occurred,” he says. “The industry generally talks of end-to-end application performance monitoring. But that excludes the end point-the user’s request to the server and the server’s response.”
Supplementing automated information
Xaffire identifies problems with a site in a couple of ways. For starters, the retailer can program the system to send an alert when certain error pages occur. “Certain type of errors contain certain words,” he says, “such as ‘database error’ or ‘sql error’ or ‘odbc.’” When tech support receives such an alert, a staffer can retrieve the problem session, then scroll through it to find where the problem occurred.
Jilk points out that the scanning program assumes the retailer knows what the problem is likely to be in the first place. So retailers need to supplement it with feedback from the call center so as to catch problems that a manager would not anticipate. The retailer needs to train the call center staff to issue alerts to the appropriate support personnel if customers are calling with complaints or phoning in their orders because they can’t complete them online, Jilk says. The tech staff then needs to obtain some identifying information from the call center reps to start diagnosis of the problem.
Support staff can search on any piece of that information, including keywords, to cull possible problem sessions from the log. Then they flip through the various possible sessions until they find the problem. “There’s a certain element of manual process here, but you have to compare it to having no idea at all what happened to these users,” Jilk says. “It’s so drastically easier than what they have had to do before.”
Whether the problem comes to tech support’s attention from the scan of the site or the call center, the next job is to isolate the problem. That occurs as the support staffer replays the session looking for the problem transaction. When he spots it, he exposes and analyzes the source code, zeroing in on the problem.
While the Xfire product is designed specifically for the IT operation of an organization, the problems that it identifies are not always technical problems, Jilk notes. “A user error could be a design issue,” he says. “In fact, there are some who believe there is no such thing as user error. It’s bad design.”
Entry level price for the Xfire product, which is a hosted solution, is $25,000, which will support two web servers. Additional servers are $5,000 each. Xaffire is rolling the product out now after development trials for several months. It is live with La Quinta hotels’ web site and the company reports it is in serious talks with several major retailers.
Capturing and filtering data