Posted 08 June 2007 - 09:39 AM
I've never done this with a www/non-www domain location because I've never had the need, but I do have a question.
What would happen if the engines (any of them) end up merging the www/non-www for their purposes, as they normally do given enough time? If they ended up merging everything to the www version of the domain and it was the one where the robots.txt was set to exclude everything, would this cause the engines to drop the site entirely from their SERPs?
I'm not sure. I can see how one could make an argument going either way. But I'd want to be pretty careful with it not knowing how it may look in the eyes of the search engines.
Frankly though, I don't see another way out of the box given that the www and non-www both need to remain active. At least not without doing some cloaked redirects when a spider user-agent was detected, which could also be pretty dangerous.