Changes Afoot at Yahoo

Search engine marketers have long been awaiting changes at Yahoo and MSN. Finally, signs of transition are beginning to appear.

Yahoo has begun crawling the web with a new indexing robot -- foreshadowing some interesting potential developments as it gets ready to drop Google as its primary search partner.

The new robot is called Yahoo! Slurp, maintaining consistency with the Inktomi Slurp crawler, purchased by Yahoo more than a year ago. Although the crawler has many similarities to the Inktomi crawler, there are some interesting new features described on the help page for the robot.

One is that the crawler is collecting documents from the web to create a "searchable index for search services using the Yahoo search Engine. For months, speculation on various search engine forums has focused on Yahoo's presumed switch from the Google index to the Inktomi index.

But other features described on the help page suggest something more is afoot.

For example, although the crawler is functional (and if you have a web site, you can see its footprints in your logs), Yahoo says that crawled pages may not show up in search results. Why? According to the Yahoo Search FAQ, "The documents will be indexed and included into the search database in the near future."

Yahoo is also now creating a cach of web pages similar to Google's.

What else is changing at Yahoo? The company won't comment yet.

Meanwhile, over at MSN, LookSmart listings have been dropped, and Inktomi results have been promoted. MSN has also rolled out a new search beta.