SEO Expert Blog

Looking at the logs of a Drupal site I just noticed that Yahoo lists XML Sitemaps in SERPs. On that site the XML Sitemap Drupal module is installed, which automatically generates an XML Sitemap from your content and if you wish also includes the category pages. There are some other quite useful configuration options the module offers and it can be set to write log messages in Drupal's watchdog table.

Using the robots meta tag as an example official Google blogger Vanessa Fox explains how Google handles Meta tags. We learn that Googlebot is not case sensitive and follows the most restrictive instructions in case of conflicts.

Yahoo! Slurp now also recognizes wildcard characters in the robots.txt file enabling webmasters to restrict what gets indexed in a more sophisticated manner.

You can use * to match a sequence of characters and $ as an anchor to indicate the end of a URL string. You can find examples in the Yahoo! Search blog.