Guest post by Joost de Valk.
Every now and then when you search, whether it be Google, MSN, or Yahoo -- RSS feeds show up in the SERPs. For instance, if you search for the popular blog boing boing on Google, the third result is the FeedBurner feed of that particular blog. This, of course, is a "bug." Most of us don't want feeds listed in the SERPs. It does however, make you think:
Feeds are bound to rank for some terms unless they are excluded somehow.
I won’t go in to why search engines seems to be indexing feeds -- fact is: they do. The feed for my personal blog has PageRank 4 at the moment, which goes to show that Google even assigns some weight to it. Now think about it, wouldn’t it be cool if you had the equivalent of a noindex, follow (not nofollow) robots meta tag for RSS feeds? That way, the feed could be followed, search engines could spider and assign weight to the links within, yet it wouldn’t show the contents of your feed in the SERPs. It would also allow controlling what gets excluded on a file-level without access to robots.txt -- sometimes a problem at larger organizations and with multiple feeds at various URLs (WordPress, anyone?). These are some of the same reasons that were posited for the creation of the robots meta tag.
This could, of course, be accomplished quite easily. For instance, PICS has been added to RSS 2.0 as an optional channel element. This basically means an extra element <rating> in the header of an RSS feed. Programs can use this rating to check if a user should be shown the content of this feed. You could also add a <robots> tag in the same fashion, which would then look something like this:
<robots>noindex, follow</robots>
This relatively simple addition to the protocol can -- if the search engines agree to it, augment what you can do with RSS feeds from an search engine marketing perspective. At the same time, this will give webmasters an easy way to solve the problem of their feeds showing up in the search results.
January 25th, 2007 at 2:26 pm
I disallow them with robots.txt or rel="nofollow".. I agree with you that's is really annoying to get feeds in SERPs
January 25th, 2007 at 3:41 pm
[...] SEO Egghead by Jaimie Sirovich » Noindex, follow for RSS Feeds? I won’t go in to why search engines seems to be indexing feeds — fact is: they do. The feed for my personal blog has PageRank 4 at the moment, which goes to show that Google even assigns some weight to it. Now think about it, wouldn’t it be cool if you had the equivalent of a noindex, follow (not nofollow) robots meta tag for RSS feeds? That way, the feed could be followed, search engines could spider and assign weight to the links within, yet it wouldn’t show the contents of your feed in the SERPs. [...]
January 26th, 2007 at 5:39 am
[...] I did a guest post over at my SEOEgghead.com asking for noindex, follow for RSS feeds. Go check it out! [...]
January 26th, 2007 at 6:14 am
Isn't this a problem that is caused by a wrong configuration of the server? If you serve your RSS feeds with the mime-type "text/xml" Google can read them perfect. But if you serve them with the right mime-type "application/rss+xml" Google doesn't index it and RSS readers can deal with them perfectly.
A robots element in the RSS feed is only a work-around, isn't it?
January 26th, 2007 at 8:05 am
rxbbx: it sure is... having to block them sucks as well though, since they do have some weight, and people WILL link to your feed...
January 26th, 2007 at 11:53 am
It's a good way to get yourself excluded from search.msn.com/feeds
or don't u want to be up there ?
i think the se's can easily fix this internally, let's submit a bug report
January 27th, 2007 at 12:18 pm
@Joost
If I click on the subcribe feed of this site, I see he has allready a pr2. I block those normal feeds. I dont really see Feedburner as such big problem in SERPs.
Regards
January 29th, 2007 at 1:01 am
[...] I notice people talking about reasons to noindex your feed and thought I would chime in with my disagreement. If you are an index junky like myself you might have noticed that your RSS feed sometimes appears in Google instead of the actual page or post (as you see in the following image below): [...]
January 29th, 2007 at 7:58 am
Both FeedBurner and other feeds in the SERPs result in another user-experience than you wanted to offer the people coming to your site from the SERPs, that's my main point... Blocking them would be foolish, as you would be throwing away perfect link equity.
January 31st, 2007 at 1:24 pm
If you don't use an outside RSS provider such as feedburner can't you just block the spider's the usual way via Robots.txt?
February 2nd, 2007 at 11:09 am
[...] I did a guest post over at my SEOEgghead.com asking for noindex, follow for RSS feeds. Go check it out! January 26th 2007 Posted to SEO, RSS [...]
February 6th, 2007 at 11:23 am
Might this affect rankings on Google blogsearch, Technorati, etc?
March 8th, 2007 at 9:55 am
[...] I am adjusting the RSS coding so that the mime-type = “application/rss+xml”. A good comment posted on SEO Egghead’s blog recently. [...]
April 15th, 2007 at 3:14 pm
[...] As regular readers might recall, I wrote a guestpost on SEOEgghead asking for an equivalent of “noindex, follow” for RSS feeds. Now it seems the search engines have been listening a bit, or at least have been thinking in the same direction… [...]
April 16th, 2007 at 4:29 pm
There's a "NoIndex" feature on feedburner under the publicize section...not sure if it also does NoFollow (which would be bad)...
April 24th, 2007 at 2:16 pm
Adam, do you have a link for the feedburner feature? Or, what I really want -- does anyone know the syntax for adding the noindex element to a feed?
May 21st, 2007 at 4:29 am
Hi John,
I wrote a WordPress plugin for it, there's an explanation of the tag in the page for it on my site.
June 14th, 2007 at 10:31 am
I've just been looking into this and I agree a noindex, follow command would be good but what about a simple method of performing a 301 [for robots only] from the feed location e.g. /feed/ say back to the home page. Or, would that get abused for SEO purposes ?
August 20th, 2007 at 5:31 pm
[...] regular readers might recall, I wrote a guestpost on SEOEgghead asking for an equivalent of “noindex, follow” for RSS feeds. Now it seems the search [...]