Keynote mobile index sets a new record for retailers with perfect availability

Eight retailers’ mobile sites were available 100% of the time. Among them was Office Depot, which jumped from No. 24 in the mobile commerce performance index two weeks earlier to No. 3 for the week ending Feb. 16.

Katie Evans

It’s been a topsy-turvy few weeks for Office Depot Inc.’s rankings in the Keynote Mobile Commerce Performance Index. In the most recent index of performance of 30 mobile commerce sites, Office Depot was up—by a lot. Eleven spots to be exact, going from No. 14 to No. 3. The week earlier, the week ending Feb. 9, Office Depot also surged upwards, climbing from No. 24 to No. 14. However, that was a big turnaround from the prior index where the retailer, No. 17 in the 2014 Internet Retailer Mobile 500 fell from No. 10 to No. 24.

Why all the back and forth? Much of the movement comes down to availability, says Joe Flake, web and mobile expert for Keynote Systems Inc. The retailer’s mobile commerce site availability, that is the percentage of time the mobile home page loads correctly, moved from 97.87% for the week ending Feb. 9 to a perfect 100% in this most recent analysis. “Page size has been pretty stable through this, with its average page load time only varying a few one hundredths of a second,” Flake says

Several other retailers on the index also achieved the perfect 100% availability for the week.  Besides Office Depot, others that were available on all tests were J.C. Penney Co. Inc., HSN Inc., Best Buy Co. Inc., Rakuten.com Shopping, eBay Inc., American Eagle Outfitters Inc., and Dick’s Sporting Goods Inc. That represents a new high for the index, Flake says.

Toolfetch.com topped the Keynote Mobile Commerce Performance Index for the week ending Feb 16. Its load time was 2.38 seconds and its success rate was 99.84% for a score of 983. Its mobile home page contains six elements weighing 63 kilobytes. Sears Holdings Corp. came in second with a load time of 2.85 seconds and a success rate of 99.85% for a score of 960. Its mobile home page contains seven elements weighing 67 kilobytes.

Two of the 30 retailers on the index offer responsive web design sites, which use a single code base and responsive techniques to adapt to the size of the device the consumer is using. This saves a retailer having to build separate sites for desktops, tablets, smartphones and other devices. Fab.com, one of those responsive sites, came in at No. 25 with a load time of 12.70 seconds and a success rate of 99.66% for a score of 678. Fathead.com LLC, the other responsive site, came in at No. 28 with a load time of 17.75 seconds and a success rate of 99.84% for a score of 482.

The average load time for all 30 retailers was 7.71 seconds, the average success rate was 99.52% and the average score was 794.

Click here then click on Keynote Mobile Commerce Performance Index Part 1 and Part 2 to view complete results for all 30 retailers on the index. Keynote measures, exclusively for Internet Retailer, 30 standalone m-commerce sites optimized for smartphones and two responsive design sites, which are single sites that render content in ways that fit the screen size of a device, including desktop PCs, tablets, smartphones and smart TVs. For the index, Keynote measures the smartphone versions of the responsive sites.

The 30 representative sites include merchants in multiple categories and channels, and of multiple sizes, ranging from such giants as Amazon.com Inc. to mid-sized retailers like Toolfetch.com LLC. Keynote tests the sites in the index every hour Monday through Sunday from 8:00 a.m. through midnight EDT, emulating two different smartphones on two different wireless networks: Apple Inc.’s iPhone 5 on AT&T and the Samsung Galaxy SIII on T-Mobile, both using 4G networks. Keynote runs the tests in New York and San Francisco.

Keynote combines a site’s load time and success rate, equally weighted, into a single score. Given that both performance and availability are important, the score reflects the overall quality of the home page; a higher score indicates better performance. Scores also reflect how close sites are to each other in overall quality. The index average score is the midpoint among all the sites’ scores. To consistently rank high on the Keynote index, sites must hit availability targets of 99.5% or better and be faster than 10 seconds to load on average. Top-performing sites load in under five seconds.

While adoption among U.S. consumers has been steadily rising, only a fraction of all U.S. wireless connections are 4G. Consequently, retailers benchmarking their mobile commerce site performance against the Keynote index should keep in mind that most of their m-commerce site shoppers will experience page load times slower than those on the index.


Keynote Mobile Commerce Performance Index, m-commerce, Mobile, mobile commerce, mobile performance management, mobile statistics, Office Depot