An A/B test of new site search technology against an older solution turns up surprises at PersonalCreations.com.
Well-engineered site search can increase the relevancy of search results, as web retailer Personal Creations Inc. recently proved it to itself on its web site, PersonalCreations.com. The gift retailer implemented a hosted site search product, Qwiser, from technology provider Celebros Inc. last fall and then ran an A/B test through January that compared results delivered by the new tool against results delivered by the standard site search function bundled into its Microsoft commerce platform.
The test revealed some surprises. “Under our old search, for example," says president of e-commerce and new business development Geoff Smith, “if a customer searched on `Mother’s Day gift,` search results would deliver every single item with the words ‘mother’ or ‘gift,’ in it.” As Personal Creations` entire inventory is gifts, such a search produced pages of questionably relevant results. “From the start, Celebros delivered much more relevant search findings. On the Mother`s Day gift search it would deliver product listings that have relevancy to moms and Mother’s Day, then provide further options for refining the search by features like category and price,” Smith says.
The log files for Personal Creations at Celebros revealed a few other surprises. Personal Creations was a cataloger before adding a web channel and Smith had believed most of the site search on PersonalCreations.com would be SKU-based versus text-based, as catalog customers visited the site with an open catalog and product SKU numbers on hand. But only 30% turned out to be SKU-based--most shoppers preferred to use text queries.
The log files also offered insights on potential product line expansion. “We found some search terms that were highly-ranked in terms of the number of searches on our site, but they were for products or categories of product that we didn’t carry. So now we are investigating adding those lines to our site,” Smith says.
Based on the preliminary test, Personal Creations already has re-optimized its site search indexing. On a new round of testing in progress now, it plans to drill down to its new site search’s effect on specific metrics such as conversion by search term, Smith says.
“The new test will do two things,” he adds. “It will show us the value of search as an enhancement tool and it will identify any area where we may have a weakness in terms of the relevancy being returned by different search terms.”