SEO Egghead Consulting Group

This handy set of rules for mod_rewrite automatically redirects any URL ending in "/index.php" to the same URL ending in "/."  It works for the root directory as well as subdirectories.

RewriteCond %{THE_REQUEST} ^GET\ /.*/index\.(php|html)\ HTTP
RewriteRule (.*)index\.(php|html)$ /$1 [R=301,L]

For example, "http://www.example.com/subdirectory/index.php" would be redirected to "http://www.example.com/subdirectory/."

This eliminates one of the peskier canonicalization issues with relative ease.  It's a bit tricky because of the way mod_rewrite updates its variables during execution — hence the odd use of the variable {THE_REQUEST}.  If anyone knows an easier way or finds a problem with it, leave a comment. 

Archived; click post to view.
Excerpt: I had to write about it eventually.  DMOZ is a horribly corrupt, flawed system.  Personally, I hope it goes away.  With the evaporation in importance of the Yahoo! Directory and the closure of Zeal, the writing on the wall is no longer just writing.  It's happening.  And with AOL running it, you may also wonder why your password hasn't been released for academic research on passwords. That said, many SEOs are still a bit DMOZ-obsessed.  Even if the pages themselves drive zero traffic, the links, for reasons I do not entirely understand still do carry a disproportionate amount of link-equity. …

Archived; click post to view.
Excerpt: Elite Hosts is a smaller full-service web hosting company.  If I had to compare it to another company, I'd say it's a lot like a smaller version of Pair.  Elite Hosts has bearded Unix gurus ready for action 24/7.  Unfortunately, their marketing department may be blinded by the big Unix beard as well.  Everything is fixable, though.  So let's get started. They appear to highlight reseller hosting, but also sell directly.  This is typical these days.  Everyone wants to outsource his marketing department — and, in fact, this is a viable model.  Doing this, however, does not obviate the need…

Archived; click post to view.
Excerpt: This is totally off subject, but I promise another post today.  In the meantime, though — I can't make this up.  I'm staying in Washington D.C., visiting a friend, and I forgot to bring deodorant.  Since I have mercy on the souls of others who stand near me in this heat, I decided I'd go pick some up at the local CVS.Unfortunately, when I got there and found the deodorant, it was locked up.  Yes, the deodorant was locked up.  I had to wait 10 minutes to get someone from CVS to open the case for me.  So either there…

Archived; click post to view.
Excerpt: I'm friendly with some spammers.  Most of them are pretty nice guys.  I don't spam, and I never will — but I keep in touch. Since most spammers are really just out to make a buck and not hurt anyone, I would never expect them to mock and laugh at a spam victim.  But sadly, that's exactly what "white hat" Rand Fishkin and the "professionals" at SEOmoz did.Let's have a look:G-man writes on their blog:"you've got to understand what I currently do for a living…I scrape the search engines and create the "crap" that they really don't like to see…

Archived; click post to view.
Excerpt: It scares me slightly more than the DMCA itself that so many who should know better are totally ignorant of basic copyright law, and, even worse, benefiting from it in a way that most do not elicit.  Obviously, if the the thief charges for the content or uses it as spider-food on a network of spam sites, that is a problem.  Regardless, the author can claim an opportunity cost.  But there is another problem, and it usually involves totally lily-"white hat" sites. Assume every link on the internet to another page has a certain value in and of itself –…

Archived; click post to view.
Excerpt: I used to assume that content behind forms was never spidered.  This does not seem to be the case, as one particular form on this blog made me aware.It appears that if Google sees a form consisting only of 1 pulldown (select), it will spider the links created by submitting the form request with the various values in the pulldown.  This has a few implications:1. Google may also spider a form consisting of any control with a finite domain, such as a group of radio buttons.  It could also decide to spider forms with multiple controls having finite domains –…

Archived; click post to view.
Excerpt: I've never assumed that the "Allow:" directive was supported by all search engine spiders.  From what I know, only Google supports it.  The draft mentions it, but that's the problem — it's just a draft.  Officially, the directive does not exist.  Admittedly, it has been in the draft state since 1997!  I guess someone should do something about it, but nobody cares enough to do so. Anyway, Google's robots.txt makes the assumption that all spiders support this draft directive, and uses "Allow:" under "User-agent: *."User-agent: *Allow: /searchhistory/Disallow: /news?output=xhtml&Allow: /news?output=xhtmlDisallow: /search It's not such a big deal, but interesting nonetheless.  I simply…