In the ongoing improvements to its web site, VistaPrint takes testing contemplated changes seriously-so much so that that the online retailer of print-related products and services runs dozens of A/B split or multivariate tests on the site per three-week cycle, Jeff Prus, senior director of user experience, told attendees in a session at the Internet Retailer Web Design ’09 Conference this week.
“VistaPrint regards ongoing site optimization as a revenue-generating function, one that generates millions of dollars per year in proven business value,” Prus said. The cornerstone of optimization at VistaPrint is usability testing that identifies opportunities, diagnoses issues and hones design; as well as A/B split and multivariate testing to confirm hypotheses about visitor behavior, measure value, and “remove opinion from the design process. Testing drives virtually every change to our site,” Prus said.
That exercise sometimes reveals unexpected answers. For example, VistaPrint normally includes a selection of four items when offering product cross-sells on its site and had hypothesized that increasing that number would increase shoppers’ uptake on cross-sells. However, site testing showed otherwise-expanding the number of cross-sell choices distracted shoppers and actually decreased overall conversion, Prus said.
Testing also has confirmed some of VistaPrint’s hunches; for example, that reducing the number of editorial options in its online document design process would increase conversions for these products. As a result, VistaPrint altered the site by retaining access to advanced editing features for customers that want them, but doesn’t feature them as prominently as it had previously.
VistaPrint has established its own internal best practices for testing, which Prus described to attendees. Retailers should establish a culture of performance-based web design, he says, to encourage ideas for and prioritization of testing opportunities. Prus also advises retailers to be systematic in their approach to testing; for example, always minimize the number of changes in a single test so as to more precisely isolate their impact. Finally, Prus says, all test results should be archived to inform future testing.