Once upon a time, in a place far away, your web server is chugging along and everything is fine.  Then, all of a sudden, something terrible happens, and the database goes down. Since this is an unanticipated error condition (all of us could do better error checking …), many pages return erroneous blank pages or 404s. Or perhaps the webserver goes down altogether.  Worse still, you don't have a hot standby to replace it.  Meanwhile, Google is trying to index your pages, not finding anything, getting blank pages, pages with heaps of errors, or 404s. Let's go through the possibilities:

Returning 404s or Blank Pages:
This is the real problem.  If a server returns a 404, a search engine will de-list your pages.  If a search engine sees blank pages, or pages full of errors, it may do the same, or rankings will suffer.  This should be avoided at all costs.

Not finding anything (no connection):
This is actually more desirable than it sounds in terms of indexing. While it may be extremely unprofessional and look bad, a search engine is likely to assume that there are intermittent connectivity problems and try again later.  Your users may, however, be annoyed.  At least from a spider's point of view, though, as long as this is resolved in a day or so, there is no major problem.

So what SHOULD one do during a server crisis?

It's actually fairly simple.  A "500" error should be returned; and a custom page describing the error could be displayed.  Perhaps the site is down for maintenance.  Say it politely and provide the time it will be back up.  Or perhaps there was a national disaster as in the case of webservers in the New Orleans area last year.  That was exactly what happened to the firm I consult for, Seeger Weiss.  The server went down in the flood because the datacenter was nearby, so we put up a global error page for every URL with a polite message and waited for things to clear up. All pages remained indexed in the end, despite being down for a few days. 

To do this in PHP use this line of PHP, then add whatever content you wish:

header("HTTP/1.0 500 Internal Server Error");
echo "The bank is closed until the scary aliens leave on their space ship; sorry for the inconvenience, thank you for being a Bank of Mars customer."; 


Tell an amigo:
  • Sphinn
  • Digg
  • Reddit
  • del.icio.us
  • StumbleUpon
  • Facebook

Related posts:
Big Daddy hates Doorway Pages (and other pages too) Since BigDaddy, many have been screaming about the decreasing quality...
Does Anyone Actually Use Status Code 410 (Gone)? I was reading through the documentation for mod_rewrite, and I...
ASP.NET 2.0 Setting Dangerous for Google Indexing Authored By: Cristian Darie http://www.cristiandarie.ro/books. I'm writing this article to...
Microsoft the Most Standards-Compliant? Is the world upside down?  By way of Jake's post...