Mobile accounted for 25% of Ulta's e-commerce revenue during Q2.
With a marketing background in catalogs, Paul Goodman knows too well the agony of waiting to see the results of testing new marketing and merchandising techniques. “You have to wait a very long period during which you test and see results,” he says. “It’s very painful.”
The web, of course, provides for faster test cycles, but still not fast enough, says Goodman, a former manager at cataloger and e-retailer Lillian Vernon Corp. who is now senior vice president of e-commerce and marketing at Dale and Thomas Popcorn. “The Internet is supposed to be immediate, but it can still take a while to read results of online tests,” he says.
That’s because online A/B testing typically uses only two page variations at a time, which can take several weeks to complete a batch of tests for any particular project. But Dale and Thomas now simultaneously tests dozens of variations taken from thousands of possible page displays, enabling it to find the best display option within a single day during periods of high traffic, Goodman says.
In the process, Dale and Thomas has increased its visitor-to-order conversion rates, and it has learned some unexpected lessons regarding accepted rules of marketing and merchandising, Goodman says.
Goodman’s team, using an optimization tool from Optimost Inc., can now test on each page several variables, each matched against a separate set of values, creating potentially thousands or even millions of page variations. The system is also designed to narrow the large number of variations to the most appropriate group of 10 or 20 that can then be analyzed.
Testing many variations of its home page during last year’s holiday shopping season, Dale and Thomas found a single variation that produced a visitor-to-purchaser conversion rate 15% higher than what the retailer had figured was its best page display, Goodman says.
The new testing system also helps experts see beyond the biases they may have toward particular merchandising techniques, he adds.
When a design person on his staff suggested that a page show popcorn spilling out of a chef’s hat, for example, Goodman objected. “I was dead set against it and said it was a waste of a display,” he says. “But I was wrong. We tested in many ways with and without the hat, and every display with the chef hat worked.”
In another surprise, Goodman says, he learned that a common rule among web merchandisers-not to use red on a transaction button-was not necessarily true. “Most web usability experts say red gets people’s attention, but that you never use red on a click-here-to-order button because red means ‘stop,’” he says. “Well guess what, in our tests red beat green, which is supposed to mean ‘go.’”