- Jan. 25th, 2007
- 21 comments
Guest post by Joost de Valk.
Every now and then when you search, whether it be Google, MSN, or Yahoo — RSS feeds show up in the SERPs. For instance, if you search for the popular blog boing boing on Google, the third result is the FeedBurner feed of that particular blog. This, of course, is a "bug." Most of us don't want feeds listed in the SERPs. It does however, make you think:
Feeds are bound to rank for some terms unless they are excluded somehow.
I won’t go in to why search engines seems to be indexing feeds — fact is: they do. The feed for my personal blog has PageRank 4 at the moment, which goes to show that Google even assigns some weight to it. Now think about it, wouldn’t it be cool if you had the equivalent of a noindex, follow (not nofollow) robots meta tag for RSS feeds? That way, the feed could be followed, search engines could spider and assign weight to the links within, yet it wouldn’t show the contents of your feed in the SERPs. It would also allow controlling what gets excluded on a file-level without access to robots.txt — sometimes a problem at larger organizations and with multiple feeds at various URLs (WordPress, anyone?). These are some of the same reasons that were posited for the creation of the robots meta tag.
This could, of course, be accomplished quite easily. For instance, PICS has been added to RSS 2.0 as an optional channel element. This basically means an extra element <rating> in the header of an RSS feed. Programs can use this rating to check if a user should be shown the content of this feed. You could also add a <robots> tag in the same fashion, which would then look something like this:
This relatively simple addition to the protocol can — if the search engines agree to it, augment what you can do with RSS feeds from an search engine marketing perspective. At the same time, this will give webmasters an easy way to solve the problem of their feeds showing up in the search results.