November 17, 2010, 11:25 AM

Making site testing a basic instinct

“Go with your gut” doesn’t translate into a better site, a design conference speaker says.

Mark Brohan

Research Director

Lead Photo

Paul Kalemkiarian Jr.

Using “a go with your gut” feeling doesn’t always translate into successful web site design. Instead retailers that have an instinctive hunch that a new design or feature will deliver better e-commerce business results need to first really test the concept, says Paul Kalemkiarian Jr., president of direct marketer Wine of the Month Club.

Kalemkiarian, who will be speaking at the Internet Retailer Web Design and Usability Conference in a session entitled “Web design smackdown: "How good is your design instinct vs. testing?” from 2:45 p.m. to 3:15 p.m. on Feb. 15., says testing—especially A/B tests of key merchandising pages and product landing pages—was critical in the redesign of in September 2009.

Over the years, the e-commerce site of Wine of the Month Club, developed new design elements based on an ad-hoc and instinctive process. “We just kept bolting on,” says Kalemkiarian.

But when Wine of the Month Club decided in September 2009 to redesign its web site, which generates 85% of the company’s revenue, it made testing a big part of the process. “You have to test these days to see if your concepts resonate with customers,” says Kalemkiarian. “Online shoppers are bombarded with images and brands. You need to test to see if your design ideas will really deliver a better user experience.”

 Wine of the Month Club is a small direct marketer, with total annual web sales are less than $10 million, says Kalemkiarian. But even smaller web merchants can make big and effective design changes by doing a better job of testing, he says.

 More effective A/B testing has helped Wine of the Month Club develop web page templates that make it easier to develop more personalized marketing campaigns, especially for e-mail, he says.

 A new design also makes it easier to test new ideas prior to any full-scale roll out. “I am constantly looking at what’s new online with our competitors,” says Kalemkiarian. “If we instinctively think there’s something we should be doing, we can test and roll out the concept much faster now that we have a much better designed web site.”

Internet Retailer’s editors asked Kalemkiarian  to speak because, after buying the company from his father in 1989, he has helped the club grow substantially, adding premium wine memberships as well as maintaining the Classic membership. The company relies on direct mail as the primary prospecting vehicle, but cultivates customer loyalty through podcasts, streaming video in e-mails, Facebook, YouTube, Twitter and more.


Comments | 2 Responses

  • Could not agree more--testing is the best way to to validate (or refute!) the "HiPPO," or Highest Paid Person's Opinion when it comes to site/marketing design. Small businesses, who have limited marketing budgets, are perhaps in the best position to take advantage of it, because A/B testing is often the best way to make sure that every marketing dollar is well spent! Pete Koomen President, Optimizely (

  • Mr.Kalemkiarian is absolutely right that testing helps to refine core page types, like merchandising and product templates.I think the key comment here is .." test to see if your design ideas will really deliver a better user experience." I would suggest two simple refinements that can add incremental dollars to his existing process - 1) use simple multivariate tests to look for combinations of design elements to goose existing results on product template pages (and elsewhere), and 2) further the 'bridge of relevance' by recognizing visitor attributes to deliver a segmented, targeted experience. Matching the paid search wording that drove a click to a specific dynamic landing page or creating consistency in imagery between the email and the landing page for example yields tremendous benefits for websites of all shapes and sizes. Nice work! - Pete Olson, Amadesa (

Sign In to Make a Comment

Comments are moderated by Internet Retailer and can be removed.

Not a member? Signup for free today!




Relevant Commentary


Jason Squardo / Mobile Commerce

Five tips for achieving high mobile search rankings

Searches on mobile devices will soon exceed those on computers, Google says. Retailers that keep ...


Sergio Pereira / B2B E-Commerce

Quill turns to its B2B customers for new ideas

Coming in April is a new section of that will let customers and Quill ...