Keynote Mobile Commerce Performance Index doubles in size

Keynote Systems also fine-tunes URL measurement and adds a carrier.

Bill Siwicki

Mobile and web performance management firm Keynote Systems Inc. and Internet Retailer have doubled the size of the Keynote Mobile Commerce Performance Index to offer a more comprehensive look at m-commerce site performance. The 30-merchant index includes chain retailers and web-only merchants of different sizes and from different product categories. It measures the amount of time it takes to download an m-commerce site home page (load time) and the number of times the home page loads successfully (success rate).

“The addition of more leading mobile commerce sites to the index will help to highlight the important issues and challenges regarding the establishment of mobile retail performance benchmarks from the consumer perspective,” says Herman Ng, mobile performance evangelist at mobile and web performance management firm Keynote Systems Inc.

Retailers new to the index include 1-800-Flowers.com Inc., Brookstone Inc., Buy.com Inc., Costco Wholesale Corp., CVS Caremark Corp., Drugstore.com Inc., The Home Depot Inc., HSN Inc., J.C. Penney Co. Inc., Lowe’s Cos. Inc., Macy’s Inc., Office Depot Inc., Overstock.com Inc., ShopNBC.com, Staples Inc. and Toys ‘R’ Us Inc.

Original merchants remaining on the list include Amazon.com Inc., Barnes & Noble Inc., Best Buy Co., Dell Inc., Foot Locker Inc., K&L Wine Merchants, Sears Holdings Corp., Sunglass Hut, Target Corp., Toolfetch.com LLC, Victoria’s Secret, Walgreen Co., Walmart.com and Your Electronic Warehouse.

The top three merchants on the index for the week ending April 8 are Walgreens, Your Electronic Warehouse and Dell. Walgreens had a load time of 5.40 second and a 98.93% success rate, achieving a score of 962 out of 1,000. Your Electronic Warehouse had a load time of 6.59 seconds and a 99.27% success rate for a score of 946. And Dell had a load time of 5.49 seconds and a 98.65% success rate for a score of 925.

The index average load time was 8.70 seconds and the average success rate was 98.14% for a score of 720. Click here then click on Keynote Mobile Commerce Performance Index Part 1 and Part 2 to see this week’s complete results for all 30 retailers on the index.

Keynote Systems has also made a change to the way in which it monitors m-commerce sites. It previously measured mobile domains; for example, m.URL.com. All the sites on the index now automatically redirect consumers from a conventional address, URL.com, to the mobile optimized version. Now that all the sites feature auto-redirects, Keynote can more precisely measure the amount of time it takes for an m-commerce site home page to download.

“We now capture additional page load time as a result of URL redirection,” Ng says. “Best practices would call for no more than two URL redirects. Each URL redirection requires an HTTP round-trip request between the mobile device and the retailer web server, all of which adds extra time to load the page. Retail sites with three or more redirects should definitely fine-tune the page setup to reduce unnecessary page load latency. Whenever possible retailers should make sure they use the smallest number of URL redirections, as this is an important area for optimizing performance.”

And Keynote Systems has added T-Mobile to its testing, now including all four major wireless carriers in index measurements.

Keynote Systems measures 30 representative m-commerce sites exclusively for Internet Retailer. The sites include merchants in various categories and channels, and of various sizes, ranging from such giants as Amazon.com, Sears and 1-800-Flowers.com, to midsized retailers like Sunglass Hut, Toolfetch.com and K&L Wine Merchants. Keynote tests the sites in the index every hour Monday through Friday from 8 a.m. through midnight Eastern time, emulating four different smartphones on four different wireless networks: Apple Inc.’s iPhone 4 on AT&T, the HTC Evo on Sprint, the BlackBerry Curve on T-Mobile and the Droid X on Verizon. The HTC Evo and the Droid X run Google Inc.’s Android operating system.

Keynote combines a site’s load time and success rate, equally weighted, into a single score. Given both performance and availability are important, the score reflects the overall quality of the home page; a higher score indicates better performance. Scores also reflect how close sites are to each other in overall quality. The index average score is the midpoint among all the sites’ scores.


Keynote Systems, Keynote Systems Inc., m-commerce, m-commerce site, m-commerce site performance, Mobile, mobile commerce, Walgreen Co., Walgreens