March 5, 2004, 12:00 AM

The Missing Metric

Whether a web site is successful depends on the definition of success—and then a retailer’s ability to measure to that definition.

Among the advice that consultants are freely dispensing to retail clients these days: Deliver shoppers from search results directly to a product page. Don’t make your customers who are looking for a particular product do another search for it when they arrive at your site.

Extremely good advice, but sometimes more difficult to put in place than a retail operations or marketing chief might suspect. The problem comes when retailers take steps to create a personalized shopping experience.

Most e-commerce managers today would agree that a retail site’s objective should be to be user-friendly, easily managed and profitable. Often, to achieve their goals, they implement session IDs or cookies to personalize the visitor experience and they purchase an enterprisewide content management system to deliver a single experience to users.

But managers at a fairly large e-commerce site, with a few hundred products, recently learned the hard way that focusing on personalization was hindering their ability to attract customers. Upon close examination of their operation, they discovered that session IDs, cookies and the content management system that so perfectly served visitors once they got to their web site were actually obstacles when it came to the web site being indexed by the leading search engines. They had learned the hard way that those investments often can have a counter-intuitive effect on a web site’s profitability.

Tripped by scalability

The problem stems from the explosive growth of the web and the consequent demand for qualified search results that have forced search engine technology to scale dramatically.

Search engine spiders crawl the web gathering HTML and following links, retrieving and storing information in a database to be analyzed later. But session IDs, cookies and content management systems are challenges for search engine spiders. Spiders have a hard time indexing dynamic URLs that contain session IDs for fear of indexing duplicate content.

Here’s how the problems develop: Retailers want to identify customers as they move through a site or when they return to a site. Thus, their content management systems, or in some cases, their web servers, generate session IDs that are transferred to an end user via a cookie. Once the cookie is deposited with the user, the retailer’s system will recognize this specific user when he returns to the site. This is beneficial for personalizing the end user experience and for the retailer to learn about customers’ shopping habits and patterns. But if the end user’s computer will not accept cookies, the session is tracked by a session ID that is encoded within the URL string. However, return sessions cannot be recorded when using session IDs because the individual receives a new session ID each time he returns to the site.

That creates a big problem for search engine spiders because they do not accept cookies. And since spiders crawl an individual site several times, they get a new session ID within the URL string each time. This gives the search engine spider the impression that it is crawling new pages, when in reality it is just indexing the same pages over and over again, creating duplicate content.

Big trouble

To a retailer, that could spell big trouble. If a site is not completely crawled, the amount of content available is reduced for a search engine to determine the relevance or importance of a site to a particular search query. This, in turn, dramatically reduces the number of times a site will be found by spiders and will ultimately reduce the number of visitors resulting from search.
The good news is that, by understanding the issues between content management systems and search engine spiders, web site owners can implement strategies to maximize benefits from both resources. A search engine optimization agency can provide recommendations on how to disable session IDs specifically for search engine spiders, enabling a retailer to realize the benefits derived from the content management systems. For instance, search engine optimization agencies can help retailers program their content management systems to determine when a computer accessing a site is a spider. It’s a fairly easy process that involves detecting whether the device accessing the site is using a consumer browser, such as Internet Explorer or Netscape, or if it is using some sort of spidering agent, which would indicate that a search engine such as Google is doing the looking.

When the system detects a spider, it can both disable cookies and prevent the session ID from being injected into the URL. Some content management systems have that functionality built in; all it takes is a change in programming to activate it. Others don’t, and that will require re-writing code to include that functionality.

Even though search engine companies are trying to find ways to accommodate session IDs, solutions don’t seem imminent, so search engine optimization companies recommend removing the potential issues from session IDs immediately to avoid any drop in search traffic. When hiring a search engine optimization agency, retailers should make sure the agency has the knowledge and experience to remove session IDs from URL strings, while enabling the retailer to realize the benefits derived from a content management system. For instance, retailers should ask agencies about their specific experience in working with the retailers’ content management system, either activating the appropriate procedures within the system or writing code to create the necessary enabling or disabling.

Know the whys

Before you talk to a search engine optimization agency, make sure you can tell the agency why session IDs are implemented on your site in the first place. If session IDs are not essential, for instance, if nobody at the company is making use of the data that session IDs generate, the best recommendation is to remove them completely. This will eliminate the session ID issues and will allow spiders access to crawl your site. For e-commerce sites that require cookies for checkout, first look at installing technology to detect whether an end user has cookies enabled (less than half of e-commerce content management systems/servers have this capability today), then create a pop-up window for users who don’t accept cookies, prompting them to do so. This is a very straightforward step that can be taken with a simple JavaScript prompt, and will eliminate the need to generate session IDs.

comments powered by Disqus

Advertisement

Advertisement

Advertisement

From IR Blogs

FPO

Gregory Ng / Mobile Commerce

Four shopping behaviors to test this holiday season

With more than 50% of traffic coming from mobile devices, retailers must test and optimize ...

FPO

Chad White / E-Commerce

The e-mail marketer’s holiday planning checklist: fall edition

It’s October, and time to make sure your e-mail marketing program is ready for the ...

Advertisement