If we're going to make this thread sticky and call it definitive, it might be useful to understand a bit of the
why as well as the
how. It might even lead to a slightly different solution?
First, it's probably important to realize that www.domain.com is technically the same thing as sub.domain.com, though it exists for a very different reason.
Ten or twelve years ago, the World Wide Web was just one small part of the Internet, and our fastest PC's were based on the brain-dead 386 chip. They weren't very fast and couldn't handle much of a load, so we often had to put the different parts of our Internet on separate machines. We'd put Apache on one computer, for example, and our mail server on another, and probably an FTP server on yet another. Each of the computers would respond to a different IP address, but still answer to the same domain name. We differentiated the computers with what we called, back then, a "machine name," and thus ended up with www.domain.com, mail.domain.com, and ftp.domain.com. (Anyone old enough to remember gopher.domain.com?)
Today's computers, of course, are much more powerful and we can put all of the different "parts" of our Internet services on the same box. Indeed, we often cram several hundred domains, each with its own set of parts, onto the same server. A few years ago, I argued (rather vehemently) that www was antiquated and should be ignored. Trouble is, a lot of big directories (Yahoo comes to mind) automatically list www.domain.com even if you submit a simpler domain.com, and we have a few billion people in the populace who habitually type www before any domain name. When I got tired of beating my head against that particular wall, I started recommending that everyone promote www.domain.com and get rid of domain.com. If ya can't beat 'em, join 'em.
The history, at least for me, is interesting, but the important thing to remember is that www.domain.com -- just like sub.domain.com -- is considered an entirely different entity than domain.com. A surfer or spider going to any of the three will receive the same 200 OK response code for a page request and won't readily be able to differentiate the domains.
The other important thing to realize, I think, is that in a more perfect world,
we really don't want a Redirect sitting between domain.com and www.domain.com.
If someone yells "Ron!" across a crowded room, they're probably going to get my attention. If someone yells "Carnell" across the same room, I'm going to respond to that, too. That's because the two names, though entirely different, generally point to the same person. Over-simplifying a bit, we can say that Ron is essentially an
Alias for Carnell.
If someone in that room walks up and addresses me as Randy, however, I'm probably going to look around, find Randy, and point him out as the one she's looking for (unless she's really cute, in which case I might lie.) Ron and Randy are two different names that point to two different people, so I need to
Redirect the person's request to where it can be best answered.
On 99.9 percent of all server configurations, domain.com and www.domain.com point to exactly the same spot on the disk. One is just an alias for the other. Using a redirect makes about as much sense as someone calling me Ron, and me responding by pointing at my chest and saying, "No, you mean Carnell." Aliases and redirects don't mix very well.
Creating an alias is generally accomplished at two levels. At the DNS level, we always have to create an entry for each, usually a CNAME for each, or possibly a CNAME for the primary and an A entry for the secondary. If the domain in question is on a static IP address, this is all we need to do to create the alias.
Most of us, however, share an IP address with other domains, so we have to move beyond the DNS level to the web server level to complete the alias configuration. Using Apache as an example, an entry in the hpptd.conf (stripped down and simplified) might look something like this:
<VirtualHost 192.xxx.xxx.xxx>
ServerName domain.com
DocumentRoot /home/domain/www
ServerAlias www.domain.com
</VirtualHost>
That's all we need. Anyone, surfer or spider, going to either domain.com or www.domain.com will get exactly the same pages. Our problem, however, is that the URL doesn't change in the address bar and the page requests will result in 200 OK responses regardless of which domain is being accessed. Remember, www.domain.com and sub.domain.com look exactly the same to a search engine. The spider must, at least initially, treat both domain.com and www.domain.com as separate entities, even though we know they are aliases.
Google is the first search engine to incorporate sophisticated duplicate content filters and the logic necessary to eventually merge domain.com and www.domain.com into the same entity. Unfortunately, there's some small emphasis in that sentence on the word "eventually." If you use a DNS alias or a ServerAlias, Google will usually and eventually recognize the alias relationship and only show one in their SERPs. In most cases, when that happens there is ample evidence to suggest the PR from domain.com and www.domain.com will also be combined. Sadly, it usually takes several months to determine and, if you change your content frequently (meaning domain.com/page.htm and www.domain.com/page.htm, when spidered at different times might have different content), those several months can stretch out to a year or more. Who wants to wait? And when the guarantees are also pretty darn fragile, who wants to gamble?
It would seem none of the "big boys" want to either wait or gamble. Most of the
PR 10 Sites we know about use only their www domain. If you try to access w3.org or adobe.com, you'll find your browser immediately redirected to the www version. It's not just an alias, because your location bar actually changes to the new www domain. We have to figure Google condones this, too, and not just because the big boys are doing it. Try going to google.com if you don't believe me. (Interestingly, most of the big boys use a Redirect 302. Only a handful, like w3.org, return a 301 status code. Presumably, Google treats them the same in this instance?)
So, instead of relying on aliases and duplicate content filters, which at best will only work with Google, someone came up with the brilliant idea of redirecting one alias to another alias, which is essentially a redirect to self. The bad news is this still only works well with Google, though even with other engines it will at least discourage people from linking to both domains and should thus avoid splitting your backlinks. The worse news, though, is that to create a redirect to self, we have to jump through a few technical hoops.
Did anyone wonder why, in Randy's first post above, he suggested both mod_rewrite and RedirectMatch, but didn't mention using a more simple Redirect? Why can't we eliminate all those complicated Regular Expressions and simply add a line like
Redirect 301 / http:
//www.domain.com to our .htaccess?
We can't do it the simple way because we shouldn't be redirecting an alias and no one ever figured we would want to try. This is one of those things that is done ONLY because search engines exist. Since domain.com and www.domain.com are aliases and point to the same directory, any .htaccess directives in that directory have to apply to BOTH domains. A simple Redirect would just send our visitors in a vicious circle and eventually crash and burn.
Let's try a slightly different server configuration, one you are very unlike to ever see.
<VirtualHost 192.xxx.xxx.xxx>
ServerName domain.com
DocumentRoot /home/domain/www
</VirtualHost>
<VirtualHost 192.xxx.xxx.xxx>
ServerName www.domain.com
DocumentRoot /home/wwwdomain/www
</VirtualHost>
Notice in this configuration, domain.com and www.domain.com point to different folders on the server and therefore are NOT aliases. We can now put an .htaccess file in the /home/wwwdomain/www folder without affecting anything in the /home/domain/www folder and use the simpler Redirect command to meet our goals.
Problem solved. Except, unfortunately, we're wasting a bit of disk space and very few hosts would do this without charging for the extra resources. That's why you won't see this configuration very often.
It does, however, suggest another possibility.
<VirtualHost 192.xxx.xxx.xxx>
ServerName domain.com
DocumentRoot /home/domain/www
</VirtualHost>
<VirtualHost 192.xxx.xxx.xxx>
ServerName www.domain.com
Redirect 301 / http
://domain.com/
</VirtualHost>
We're still configuring domain.com and www.domain.com as separate entities (not aliases), but instead of defining a DocumentRoot for one, we simply insert our simple Redirect statement. We avoid the contradictions of trying to redirect an alias by never creating one.
The disadvantage to this method is that it will require more work for the host admin. He has to define two blocks instead of one, but more importantly, he has to maintain both blocks in unison. If one is removed or altered, the other needs to also be removed or altered, and this introduces the very real probability of human error. (The real reason he will balk is because none of his utilities, like Cpanel or Ensim, will automate the procedure for him. At least, not yet.)
The trick is going to be to convince your web host that the advantages TO THEM outweigh the disadvantages.
Your .htaccess file introduces overhead to the server. Every single page request made to your domain will result in a read operation for .htaccess and a parsing of the file. That constant rereading and reparsing is why you can change your .htaccess and have the changes take immediate effect. Now throw in all those Regular Expressions that have to be evaluated for every single page request, and then multiply all of that by the 200 to 500 other domains sitting on the server. Your web host is going to find they can put fewer and fewer domains on a single box if all those domains are jumping through technical hoops in order to accomplish a simple redirect.
A redirect placed in httpd.conf, however, requires no fancy RegEx evaluations and more importantly is read and parsed exactly ONCE. Apache reads httpd.conf when it is first booted, memorizes it, then ignores it until the next reboot. The processing is minimal and the load on the server even less so.
Redirecting an alias is a contradiction, and one that exists only because of the search engines. Everyone, including the big boys, including even Google, does it, but it will always be just a kludge. We need the redirect. A more elegant solution, IMO, is to avoid creating the alias in the first place. That it makes our servers run more efficiently is a nice side-benefit.
This topic is locked
Ian! Perhaps I should save it too, hunh?
Excellent lesson Ron.
