Critics have claimed responsive design site performance is slow. In a first-of-its-kind e-retail study, Internet Retailer and Keynote examined 12 retail responsive design sites and found that site load times on smartphones are not only slow - at 18 seconds, they're very slow. But some retailers and mobile commerce vendors have devised new ways to optimize responsive sites that vastly reduce mobile page load times.
Bill Siwicki , Managing Editor, Mobile Commerce
A 1-second delay in web site page load time translates into a 7% loss in conversions, according to research firm Aberdeen Group Inc. So if an e-retailer makes $100,000 a day from its mobile site, a 1-second page delay could mean around $2.5 million in lost sales every year. If that's the case, what does an 18-second page load time mean?
Unless your business model is giving away money, not even a consumer with the patience of Job is going to stick around for 18 seconds to see your wares. In fact, 79% of smartphone users will avoid shopping at sites they consider slow, according to a study by content delivery network Akamai Technologies Inc.
Retailers and experts in e-commerce and m-commerce have long maintained that responsive sites load too slowly. But there has never been an objective measure to test the merit of this complaint. Until now.
It turns out responsive design sites are slow on smartphones, very slow. That's the conclusion that emerges from an Internet Retailer-exclusive, monthlong study of 12 e-retail responsive design sites conducted by web and mobile performance testing, monitoring and analytics firm Keynote.
While the home pages of the 12 sites on average loaded on desktop PCs in 3.15 seconds and on tablets in 2.80 seconds (both on high-speed connections), it took a staggering 18.24 seconds on average for responsive home pages to load on smartphones over a combination of 3G and 4G connections, Keynote finds (see box at the end of this story for methodology). Not only is this unacceptable when it comes to performance (Keynote says optimal page load times on smartphones are 2 to 6 seconds, depending on the Internet connection speed), an 18-second page load time is a gigantic hurdle to converting shoppers on smartphones into buyers.
"Retailers know huge numbers of consumers use smartphones as part of the purchase process, but so many consumers on smartphones delay committing to purchase until they can access a different device," says Ken Harker, the senior consultant at Keynote who led the responsive design site performance test. "If retail web pages on smartphones were snappy and fast with 2-second load times, you would have a lot more people committing to buy right then and there on their smartphones rather than wait until later to complete the purchase on a device that offers far faster performance."
To put the 18.24-second responsive smartphone load time in perspective, on the weekly Internet Retailer-exclusive Keynote Mobile Commerce Performance Index, which measures 28 standalone m-commerce sites designed specifically for smartphones and two responsive sites on smartphones, the average page load time for the week ending May 4 was 11.09 seconds. The standalone m-commerce site of Sears Holdings Corp. had the top speed of 2.29 seconds.
Responsive design is heralded by some as the future of e-commerce and m-commerce site design, but it is still relatively new to e-retail, and the performance issues it causes may trouble online retail executives. But while Internet Retailer's and Keynote's first-of-its-kind study of responsive in retail shows that responsive design is not a reliable solution for serving customers on smartphones, some retailers and vendors are showing there are ways to optimize responsive sites, as well as new responsive approaches that can vastly improve performance on smartphones.
And they're keeping in mind, based on earlier Internet Retailer research on m-commerce sites for smartphones, that consumers want smartphone sites with the features and functions of a desktop site. That means hacking off parts of a site to make it load faster is not the answer. Further, retailers benefit from customers doing much of their shopping on smartphones at home, where they have high-speed Wi-Fi networks that speed up sites with even the most egregious load times on slower connections. But even on super-fast Internet connections like Wi-Fi there remains a significant difference in load time between m-commerce sites (faster) and responsive design sites (slower), which only sets the bar higher for responsive design.
Retailers have yet to discover the winning formula for employing responsive design, says Michael Facemire, a Forrester Research Inc. principal analyst who specializes in mobile and responsive sites and mobile apps.
"It's still early in retail when it comes to learning how to create mobile shopping experiences, regardless of whether it's on a responsive design site, a mobile site or a mobile app," Facemire says. "This is especially the case with responsive design."
Facemire adds that businesses building responsive sites often take shortcuts to quickly get responsive sites online.
"Mobile is still early in its maturity, yet demand for mobile experiences is incredibly high," he says. "That combination leads retailers to take shortcuts. One shortcut I see often is how fast can we get something to market—and in the process they overlook quality and performance."
Adding six weeks to the development time to be thorough beats delivering fast with a site that takes 18 seconds to load on a smartphone, Facemire says.
Responsive design is appealing because it allows a retailer to cater to consumers on all kinds of devices with a single web site. Otherwise, a retailer that maintains separate sites for PCs, smartphones and tablets has to add every product or promotion three times, a big, ongoing maintenance burden. Generally speaking, responsive design uses one code base and one set of web content to create a single site that renders in different ways to optimally fit the size of the screen on a device requesting a web page, regardless of whether that device is a PC, tablet, smartphone, smart TV or other device.
There are two general types of responsive design sites: pure and hybrid. Pure sites, which follow the original responsive design approach, cram everything needed to create all the versions of a site for desktops, tablets, smartphones and other devices into one huge package that is sent to a device. On the device side, the web browser unpacks what is needed for that device. This is also known as the client-side approach.
The hybrid responsive design approach detects the type of device requesting a page and a retailer's servers only send what is needed to build a page to fit that device's screen, thus reducing the data load and, in theory, improving performance. There is an ever-increasing number of flavors of hybrid responsive design, with vendors scrambling to improve both the design and performance of responsive sites to lure retailers intrigued by the responsive technique. This style is also known as server-side.
The 12 retail responsive design sites (10 U.S. and two U.K.) examined in the Internet Retailer/Keynote study include a blend of pure and hybrid sites. Most are of the pure variety because that style is more established. But both pure and hybrid sites exhibited poor performance.
Keynote ranked the 12 sites by averaging each retailer's three ranks, on desktops, tablets and smartphones. The top performer was Hiut Denim Co., a U.K. apparel retailer. Its home page loaded on desktops on high-speed connections on average in 0.92 seconds, on tablets on high-speed connections (the overwhelming majority of tablet use is on high-speed Wi-Fi networks, including in homes) in 0.93 seconds, and on smartphones on a combination of 3G and 4G networks in 5.73 seconds. 4G refers to the faster wireless networks that carriers are deploying to replace slower 3G connections.
HiutDenim.co.uk was the only retail responsive site to beat Keynote's optimal performance recommendations: 2 seconds for desktops on high-speed landlines, 2 seconds for tablets on high-speed Wi-Fi, and 2 to 6 seconds on smartphones, depending on wireless speed (2 seconds for 4G LTE—a faster version of 4G—3 seconds for 4G and 6 seconds for 3G).
Keynote recorded numerous measurements in its monthlong test, including two that tell the story of responsive on smartphones: the number of elements (such as images or scripts) on the home page by device and the home page weight in kilobytes by device. HiutDenim.co.uk contains 28 page elements that combined weigh 647 kilobytes. Keynote says retailers wanting fast-performing sites on smartphones should limit web pages to 50 elements weighing 500 kilobytes when on 4G LTE or Wi-Fi connections, 30 elements weighing 300 kilobytes when on 4G connections, and only 10 elements weighing 100 kilobytes when on 3G connections.
By comparison, the heaviest responsive design site home page on smartphones among the 12 retailers in the study belongs to TheCandylandStore.com. The sweets merchant's pure responsive home page contains 147 page elements weighing a hefty 2.6 megabytes. Little wonder that the TheCandylandStore.com home page took a leisurely 46.50 seconds to load on smartphones, by far the longest load time in the study.
Hiut Denim did not reply to repeated requests for comment on how the merchant manages such solid performance with responsive design on smartphones.
However, the retailer that ranked second in the responsive study, Sweetwater Sound Inc., a U.S. musical instruments and pro audio e-retailer, says its relatively quick load times are the result of it understanding its customers. The home page of hybrid responsive design site Sweetwater.com loaded on average on desktops in 1.23 seconds, on tablets in 0.76 seconds, and on smartphones in 8.74 seconds, Keynote finds. 8.74 seconds is fair in that it's not 18 seconds, but it does fall just outside of Keynote's optimal speed range of two to six seconds for sites on smartphones.
The key to a retailer getting decent performance with a responsive design site is tailoring the design to meet its customers' specific needs, says Mike Clem, vice president of e-commerce at Sweetwater, which dropped its m-commerce site for smartphones and rolled out its hybrid responsive design site in 2012. Clem refers to pure responsive design as a "one-size-fits-all" approach that carries a lot of additional, unnecessary weight.
Further, Sweetwater uses a design technique known as "lazy loading." Here, Sweetwater first loads only what a consumer can see "above the fold" on the first screen. A consumer may only need a link on the top of that page—such as a store locator—which he can tap to reach that page. Thus, there's no need to load the bottom of the original page. But if the consumer does begin swiping down that original page, Sweetwater detects that and then loads more page content.
Sweetwater's approach—designing based on what customers need— is a wise one, based on March 2014 research conducted by Retail Systems Research LLC exclusively for Internet Retailer. That study found 49% of consumers who land on a site optimized for smartphones leave that site and instead shop a retailer's full desktop site on their smartphones. Retailers and experts agreed that many retailers have gone too far in streamlining mobile sites, cutting features and functions customers need to make a purchasing decision. Many further agreed that desktop sites on smartphones is not the answer, that retailers need more robust but still optimized sites for smartphones, and that one solution to this problem is responsive design.
And that is why executives at web-only merchant eBags Inc. continue to work at striking the right balance between experience and performance. They know customers require many features and functions to decide to buy, regardless of what device they are using. In the Internet Retailer/Keynote study, the responsive design site home page of eBags.com loaded on average in 3.18 seconds on desktops, 2.37 seconds on tablets, and 26.66 seconds on smartphones.
"We have, for example, 2.9 million customer testimonials and countless images—customers need this rich information to help make a purchase decision," says Mike Frazzini, vice president of technology at eBags. "This requires a lot of bandwidth, but that is what helps a retailer get customers to pull the trigger and buy."
Regarding the 26-second load time on smartphones, Frazzini says eBags is not done working on its responsive site, which he says took 8,000 to 10,000 person hours to build.
One fix it is working on is what many are calling "responsive imagery." The eBags.com site features a large hero image on the home page, and currently it serves that image to smartphones and fits it to the mobile screen. But eBags is in the process of making its images responsive, which means that when the retailer detects a smartphone, it will send only 30% of the image, resizing it on the fly. It says this fix alone should cut load time by 60%.
"That is the low-hanging fruit to whittle down our smartphone performance number," Frazzini says. "But this fix creates a problem for the art director because you can't just arbitrarily resize images and have them look correct. You might have a big oak tree and a picnic underneath, and when you resize to 60x60 suddenly the picnic has vanished. So the art director has to make sure imagery is appropriate when resized for smartphones."
Extreme resizing of huge images not only reduces the kilobyte weight but also affects how much of the original image is seen, hence the need for art directors to be part of this process.
One of eBags' biggest problems in the mobile performance realm is page weight: Its home page on smartphones contains 123 elements that together weigh 1.16 megabytes. That's huge even for Keynote's most generous recommendation, which is that home pages contain no more than 50 elements weighing no more than 500 kilobytes for transmission over the fastest connections, 4G LTE and Wi-Fi.
Harkening back to the Internet Retailer/Retail Systems Research study that suggested retailers have been cutting too many features and functions from mobile sites, eBags says that's one reason its pages are still heavy—it's trying to find the magic spot where experience and performance strike a good balance.
"Customers respond to a richer experience, so there's a limit to how much weight we can cut," Frazzini says.
Tom Cox, CEO of Golfballs.com Inc., whose hybrid responsive design site was part of the Keynote responsive site test, agrees with Frazzini, saying retailers must be careful not to cut too much in the name of performance. Golfballs.com's hybrid site launched in February 2013 at an initial build cost of $20,000. Golfballs.com is continually working on optimizing page performance, Cox says. According to Keynote, the Golfballs.com home page loads on average in 3.57 seconds on desktops, 2.58 seconds on tablets, and 11.46 seconds on smartphones.
"We know we could speed load times by serving fewer page elements, but for optimal conversion there needs to be a balance of elements served for speed and elements that appeal to customers," Cox says. "More than 75% of products sold by Golfballs.com are customized in some way. We believe it is important to offer some level of preview functionality, which adds to the number of page elements and the page weight. But in the process of adding weight and elements, hopefully we are answering customer questions during the mobile order process, thereby improving conversion."
There's a big "but" when it comes to discussing mobile performance. In testing the 12 responsive design sites, Keynote used a blend of 4G LTE (the fastest speed via wireless carrier, it approaches Wi-Fi speeds) and 3G (a much slower speed). Only 20% of U.S. Internet-enabled mobile phone users have 4G or the faster 4G LTE wireless data connections, while 71% have 3G and 9% have 2G, according to research firm Informa Telecoms & Media. So the Keynote test reflects today's realities for smartphone users on the go.
But it's important to remember that more than two-thirds of smartphone shoppers use their mobile devices to shop in the home at times; further, 72% of smartphone shoppers who make a purchase with their mobile devices mostly do so in the home, research firm The Nielsen Co. finds. And according to research and consulting firm Deloitte, 63% of U.S. smartphone users most often connect to the web on their devices on a Wi-Fi network.
So what does that mean? Suddenly those molasses-like 26-second page load times speed up—way up. Keynote conducted a test of two responsive design sites from the weekly Keynote Mobile Commerce Performance Index. Measured on a blend of 4G (not LTE) and 3G connections on multiple smartphones, Fathead LLC's home page loaded on average in 16.75 seconds and Fab.com's home page loaded in 17.58 seconds. However, measured entirely on the super-fast 4G LTE, Fathead came in at 6.69 seconds and Fab.com at 6.70 seconds. The loads on Wi-Fi would be even quicker.
Suddenly, responsive design site performance is not looking so bad.
But there's another "but." Actually two. First, Keynote conducted the same test it did with Fathead and Fab.com with two retailers that offer standalone m-commerce sites for smartphones, Toolfetch.com LLC and W.W. Grainger Inc. Grainger's mobile home page load went from 2.65 seconds to 1.02 seconds, and Toolfetch.com's mobile home page load time went from 2.11 seconds to 0.81 seconds. Consumers who have 4G LTE access compare 0.81 seconds (non-responsive) with 6.70 seconds (responsive)—and suddenly the performance bar has been set much higher, and responsive design once again is slow.
Second, as the studies show, a shopper on a smartphone spends around two-thirds of her time connected to the Internet by Wi-Fi. Let's say she's patient—to her, 0.81 seconds or 6.70 seconds are totally fine. However, say that same shopper has a smartphone with a 3G wireless data plan, and she spends one-third of her shopping time on 3G. Suddenly that 6.70 seconds explodes to 17.58 seconds. While patience is a virtue, there are few shoppers who will stick around for almost 18 seconds waiting for a page to load—shoppers will quickly pivot to that painfully slow responsive design retailer's competitors, and possibly never return on mobile to the slow retailer, Keynote's Harker says.
Vendors are keenly aware of responsive design's performance shortcomings, but also the promise of efficiencies and control the technique offers. So some vendors have been busy creating new flavors of responsive design that specifically attack the slow load time problem. (See boxes on Pioneering Vendors at the end of this story.)
When mobile commerce platform provider Moovweb Inc. heard about the Internet Retailer/Keynote responsive study and the awful results on smartphones, the vendor told Internet Retailer it could take one of the 12 sites studied and make the site's performance on smartphones far better using its so-called "responsive delivery" technique. Internet Retailer selected Tattly.com as the guinea pig, Keynote agreed to recreate the smartphone aspects of the study to retest Tattly.com, and Moovweb went to work. Three days later Moovweb had its versions of Tattly.com pages ready to go (but on secure servers so consumers and search engines could not access the site, only Moovweb and Keynote).
For four days (less than the month in the original Internet Retailer/Keynote test), Keynote tested the Moovweb version of Tattly.com and a control version of the same pages on Tattly.com as the site appeared in its regular state when Moovweb secured all the necessary data and page elements.
In "responsive delivery," Moovweb splits the front end of a desktop site from the back end of the desktop site and sends the front-end code and content through what the vendor calls its "transformation engine," a server that uses responsive design techniques to optimize pages for display on smartphones. Back-end systems and processes stay the same.
Moovweb's responsive delivery version of Tattly.com on smartphones was 61% faster than the control site—17.50 seconds versus 44.87 seconds, Keynote finds. What's more, the Moovweb version was 65% faster on only 4G wireless connections, bringing response time down from 17.14 seconds to 6.0 seconds, Keynote says.
The Moovweb platform, as part of "responsive delivery," quickened page loads by automatically optimizing large image files into responsively sized, device-specific images and caching commonly used static page elements, such as headers and footers, for repeat use, among other techniques. Moovweb reduced Tattly.com's home page weight by 74%—but even then, the optimized page was 800 kilobytes, still too heavy for 3G in Keynote's view.
But Moovweb says if Tattly.com were a client, it could make changes to the responsive site that would speed loading even more. The vendor cites its client 1-800-Flowers.com Inc., which has a responsive delivery site: On 4G, Moovweb cut the merchant's average page load time by 30%, from 6.8 seconds to 4.7 seconds; and, significantly for slower connections, cut the time to start rendering a page by 60%, from 2.2 seconds to 0.9 seconds, says Amit Shah, the retailer's vice president of online, mobile and social.
"The customer can begin viewing the page in a second, and that's the customer's perception of performance," says Moovweb product manager Miheer Bhachech. "Generally, on slow networks, which were generating those huge full-page load times, we see measurements like 'time to first paint' coming in at only 30% of the full page load time. At 30%, consumers can start interacting with a site. That's absolutely key."
The first-generation responsive design sites in retail today simply do not perform well on smartphones when looking at a blend of the common 4G LTE, 4G and 3G wireless carrier connections, Keynote's data shows. Though the majority of smartphone shopping might be at home on Wi-Fi, retailers whose responsive design sites take a long time to load on slower wireless connections are writing off a huge chunk of consumers. That's not an option if a retailer wants to be successful in the increasingly mobile world of Internet retailing,
Some retailers with first-generation responsive design sites come close to good smartphone load times, as HiutDenim.co.uk and Sweetwater.com show. But retailers should keep their eyes on pioneering developers and vendors (see boxes on Pioneering Vendors at the end of this story) that are evolving responsive design in ways that greatly reduce long page load times while still offering feature-rich shopping on smartphones.
Retail responsive site design study methodology
Bill Siwicki, Internet Retailer's managing editor of mobile commerce, selected 12 retail responsive design sites (10 in the United States and two in the United Kingdom) across a number of product categories and responsive styles (pure and hybrid). Mobile and web performance testing, monitoring and analytics firm Keynote then tested the sites for the entire month of February 2014. It tested the sites on desktops on high-speed connections, testing the U.S. sites from 11 locations in the United States and the U.K. sites from four locations in the United Kingdom. Keynote tested the sites on tablets on high-speed connections (since the vast majority of tablet use is on high-speed Wi-Fi networks), the U.S. sites from two U.S. locations and the U.K. sites from two U.K. locations. And it tested the sites on smartphones on a blend of 3G and 4G wireless carrier connections (reflecting common wireless data plan speeds today), the U.S. sites on all four major wireless carriers (AT&T, Sprint, T-Mobile and Verizon) from eight locations and the U.K. sites on four major U.K. wireless carriers (O2, Three, T-Mobile and Vodafone) from four locations. Keynote used Internet Explorer 9 on desktops, the iPad 3 for tablets and the iPhone 5 for smartphones. It conducted measurements every 15 minutes on desktops and tablets and every 60 minutes on smartphones.
PIONEERING VENDOR – Yottaa
Breaking responsive HTML pages into small fragments, delivering only what is needed when it's needed. Sequencing the rendering of different page elements, showing important ones first. Resizing, compressing and sequencing images.
The Yankee Retail Company LLC, Home-Decorating-Co.com
4G: 13.3 seconds to 5.9 seconds
Wi-Fi: 8.4 seconds to 3.2 seconds
Page views per visit: +20.9%
Click here to read a profile of Yankee Retail's work with Yottaa.
PIONEERING VENDOR – UniteU Technologies Inc.
City Sports, CitySports.com
3G: 6.02 seconds
4G LTE: 2.12 seconds
Cable modem: 0.86 seconds
Click here to read a profile of City Sports' work with UniteU.
FEATURED RESOURCE PROVIDERS OF E-COMMERCE TREND SETTERS
Enhance your store's sales power with high-end branding, merchandising, and usability solutions from EYStudios! We'll show you how a custom responsive design can wow your customers and beat the competition.