November 2, 2010, 11:16 AM

The spider is always right

Higher natural search rankings can come from better treatment of search crawlers.

Mark Brohan

Research Director

Lead Photo

Mark Carson

To do the best job possible with web site design and search engine optimization, online retailers need to think of search crawlers the same as they would their best customers, says vice president of marketing Mark Carson.

Carson, who will speak about the challenges of designing better optimized pages at the Internet Retailer Web Design & Usability Conference in a session entitled “Joined at the hip: Site design and search engine rankings on Monday, Feb. 14 from 3:15 p.m. to 4:15 p.m., says redesigning a web page isn’t just about presenting a better user experience.

Equally important to web retailers is designing pages that improve natural search results by treating search engine spiders—the crawlers that find and grab content from web sites and add it to search engine indexes—as another customer set.

“Just as you think about how a customer wants to use the site and take that into account when redesigning a web site, the same holds true for designing for what the spider wants to see,” says Carson. “Think of the crawler as one of the customers your designers want to please the most.”

By paying attention to how the spiders from Google, Yahoo, Bing and other search engines crawled and indexed content, the online retailer was able to redesign its pages and improve its natural search results by eliminating unnecessary Flash elements on the home page and on key merchandising pages, and introducing more focused keyword links. To improve natural search rankings, also added more specific content such as a series of blogs and more textbook and product-specific keywords.

“A spider can’t see through Flash and text links have to be placed on the page in a way that’s natural for the spiders to find them,” says Carson. “Better optimization means looking at design from the spider’s point of view.”, No. 246 in the Internet Retailer Top 500 Guide, introduced a major redesign in July that features enhanced navigation, better product recommendations and the addition of social sharing tools. To enhance site optimization, designers and marketers also boned up on the latest innovations Google was making to how it indexes content.

“We wanted to understand how Google was collecting the content so we could display the content,” says Carson. “We looked at our keyword usage and density, studied how Google was collecting it and made several optimization improvements based on what we found.”


Sign In to Make a Comment

Comments are moderated by Internet Retailer and can be removed.

Not a member? Signup for free today!




Relevant Commentary


Jason Squardo / Mobile Commerce

Five tips for achieving high mobile search rankings

Searches on mobile devices will soon exceed those on computers, Google says. Retailers that keep ...


Sergio Pereira / B2B E-Commerce

Quill turns to its B2B customers for new ideas

Coming in April is a new section of that will let customers and Quill ...