Weekly Keynote mobile performance index goes 4G

The performance firm is using new technology to measure 30 m-commerce sites.

Bill Siwicki

Web and mobile performance management firm Keynote has upgraded the technology behind its weekly Keynote Mobile Commerce Performance Index, which measures the page load time and page load success rate for 30 m-commerce sites and ranks the sites by performance quality. The index, provided exclusively to Internet Retailer, previously measured sites via 3G wireless network connections; now it is using 4G wireless connections. 4G networks offer data speeds significantly faster than 3G networks; speeds vary widely depending on a variety of factors surrounding the quality of a wireless connection, Keynote says.

"Keynote recently upgraded its Mobile Web Perspective service used to measure the end-user perspective of mobile web site visitors under the newest 4G technology on the AT&T and T-Mobile networks," says Abelardo Gonzalez, mobile performance expert with Keynote. "The locations where the index is being measured from, San Francisco and New York City, are two where the carriers are now using 4G as the preferred network for any new subscribers. Given that we have upgraded our hardware to support 4G networks, Keynote decided to use this service to benchmark the index given 4G will be the standard for the next generation of wireless data experience."

While 4G adoption among U.S. consumers has been steadily rising, only a fraction of all U.S. wireless connections are 4G. Consequently, retailers benchmarking their mobile commerce site performance against the Keynote index should keep in mind that most of their m-commerce site shoppers will experience page load times slower than those on the index.

Sears Holdings Corp. topped the inaugural 4G version of the Keynote index for the week ending Sept. 15 with an m-commerce site home page load time of 2.84 seconds and a page load success rate of 99.16%. Weighting and combining load time and success rate give Sears an index score of 998 out of 1,000. Toolfetch.com LLC came in second with a load time of 3.51 seconds and a success rate of 99.18% for a score of 989. And W.W. Grainger Inc. came in third with a load time of 5.32 seconds and a success rate of 99.49% for a score of 967.

The average load time for all 30 retailers on the index was 12.73 seconds, the average success rate was 95.35%, and the average score was 841.

Click here then click on Keynote Mobile Commerce Performance Index Part 1 and Part 2 to view complete results for all 30 retailers on the index.

Keynote measures 30 representative m-commerce sites exclusively for Internet Retailer. The sites include merchants in multiple categories and channels, and of multiple sizes, ranging from such giants as Amazon.com Inc. to mid-sized retailers like Toolfetch.com LLC. Keynote tests the sites in the index every hour Monday through Sunday from 8:00 a.m. through midnight Eastern time, emulating two different smartphones on two different wireless networks: Apple Inc.’s iPhone 5 on AT&T and the Samsung Galaxy SIII on T-Mobile, both using 4G networks. Keynote runs the tests in New York and San Francisco.

Keynote combines a site’s load time and success rate, equally weighted, into a single score. Given that both performance and availability are important, the score reflects the overall quality of the home page; a higher score indicates better performance. Scores also reflect how close sites are to each other in overall quality. The index average score is the midpoint among all the sites’ scores. To consistently rank high on the Keynote index, sites must hit availability targets of 99.5% or better and be faster than 10 seconds to load on average. Top performing sites load in under 5 seconds.


4G, Abelardo Gonzalez, Keynote, Keynote Mobile Commerce Performance Index, m-commerce, Mobile, mobile commerce, mobile performance, mobile statistics, Sears Holdings Corp., Toolfetch.com, w.w. grainger inc.