10 replies to this topic

[Matt]

Hello

I have seen this question on a few forums before, but there seems to be so many mixed answers, but I am faced with moving a site that has done really well in rankings and is being spidered by all those that count.

Will moving a site to a new server (new IP address) result in:

any loss of rankings?
lose the ability to be spidered?
any other SE problems?

Thanks and regards
Matt

[Scottie]

Welcome to the forum Matt!

[Scottie]

Will moving a site to a new server (new IP address) result in:

any loss of rankings?
lose the ability to be spidered?
any other SE problems?

If you were changing domain names, you would have more to worry about. Just moving the site to a different IP address won't hurt anything- when Googlebot or a visitor requests your domain, the new IP address will be provided and that's where it will go. There shouldn't be any problems.

[Jill]

If you are changing domain names, simply permanently redirect (301) the old domain name to the new one.

Jill

[Ron Carnell]

Just moving the site to a different IP address won't hurt anything- when Googlebot or a visitor requests your domain, the new IP address will be provided ...

Ah, if only life were that simple.

Everyone knows that when you move a domain to a new IP address, you inevitably face the issue of DNS propagation. The Domain Name System is a distributed database, and your new information has to be essentially distributed around the entire world. It can take a few hours, a few days, or up to a whole week before everyone finds you on the new server.

The main culprit in DNS propagation is caching. When you click on a link to a new domain, your named server (assigned by your ISP in most cases) talks to a Top Level named server (there's at least one for every domain extension), who in turn talks to the named server handling your domain (usually assigned by your hosting company). After jumping through a few hoops, the domain name is resolved to an IP address, to which your browser can made a connection.

What happens if you return to that same domain tomorrow? Your browser talks to your local named server (the one assigned by your ISP), but yesterday's long conversation between computers is no longer necessary. Your local named server has a cache of that conversation and should be able to immediately return the correct IP address to match the domain. DNS caching is vital to limiting unnecessary Internet chatter, but is also the primary cause of delaying propagation when you move to a new server. Until that local named server clears its cache and again initiates that long conversation between computers, it has no way to know you've moved to a new IP address. The length of time it maintains its cache is the single biggest factor in DNS propagation (you can control this to some extent by correctly setting the TTL parameter on your old server before the move).

Because of DNS propagation, no one in their right mind would immediately turn off their old server after moving a site. Instead, they keep both sites live for a week, knowing some visitors will hit the old site and others, as the domain change propagates around the world, will start hitting the new server.

This is all old news, of course.

However, there is rather strong evidence to suggest that Search Engine spiders follow a very similar procedure of caching your IP address the first time they visit your site. This only makes sense considering how expensive it is to resolve a domain name, in terms of time and resources. Every once in while, they clear their cache, do a new DNS lookup, and then cache that IP for their next few visits. No one really seems to know how long they keep the cached IP address. Could be a week, could be a month.

What happens if the spider visits that old IP address and doesn't find your site? Four or five years ago, pre-Google, moving to a new IP address without taking certain precautions was a guaranteed way to get dropped from the index for a month. The now defunct Excite SE was notorious for this, although AV was almost as bad. I always recommended keeping both sites live for at least a month, which seemed to be long enough for the spiders to clear their cache and find the new server. It cost a month's hosting fee, of course, but that was the only downside. All of your visitors saw the new site as soon as propagation completed, but the old site was still there for the spider to weave its web.

I haven't moved a site in almost two years, so my personal experience is limited. Googlebot is, by far, a lot smarter than the old spiders, but I honestly don't know what is required to trigger a new DNS lookup and, frankly, wouldn't take a chance on being dropped. I'd pay the extra month of hosting fees, set up a 301 redirect as Jill suggests, but otherwise keep both sites available for the spider. I think I'd likely sleep a whole lot better at night.

[Jill]

Keeping the old site up for 1 month is definitely good practice. There have apparently been recent cases of Google caching the old dns and sites being inadvertantly removed. I believe this is rare, but it has happened.

You can also watch your logs at the new server and once you see the search engine spiders crawling around it, you know it's pretty safe to delete the old files from the old server.

Jill

[Matt]

Thanks guys for your replies

Ron, I share your concern

Now, that's my whole concern i.e. the spider still looking at old DNS - finding nothing and then all hard work is undone.

I hear what you are saying but practically, does it work like this:

site moves to new server, so new DNS records are established
on the old server, I just keep all the files as they are for about a month till some eveidence that new sites are spidered, then delete them.

so no users are able to access the old server site, the files are just sitting there under an IP address, while spiders are (hopefully) doing there stuff on the new server.

Thanks, this is helpfull and why take any risks if you have done a lot of work, spent a lot of time getting good rankings...

[Jill]

Matt, sounds like you've got a good handle on things now.

Good luck!

Jill

[fred]

Is google only using dns to revisit your site

I read a while back , I don't remember where ( i tought it was on google site ) that google use ip adresses when revisiting for improved speed reason. meaning that the bot doesn't have to go through the dns lookup and going directly to the site's ip adress.

Is that true, was that true or am I seeing things ?

Fred

[Ron Carnell]

You've got it nailed, Matt. Assuming it's a shared server, I wouldn't even tell the host I had moved until I was ready to pull the plug (Apache does weird things with HTTP 1.1 requests if the domain isn't defined in httpd.conf). As long as the domain has propagated, you don't need to worry about visitors finding the files (unless, as you said, they used the IP address). I usually make a minor change in a main file and then search within the site to see if that change has been indexed. Today, with Google's cached pages, it would be even easier to determine.

Fred, it's not quite that simple, but almost. Yes, it is almost certain that Google (and all other major spiders) use the IP address to revisit a site. It only gets slightly more complicated because they don't use just the IP address. There are too many sites sharing an IP address, so the domain is still important. For this discussion, though, we can ignore that, because the domain is only important AFTER the spider reaches the right (or possibly wrong) IP address.

[projectphp]

I had this issue recently, so I dug around a little, and found some good info.

http://www.webmaster...forum3/9071.htm has a similiar experience, and the figure quoted is like two months.

http://computer.hows...rch-engine1.htm talks about Google's own Cached version of DNS records.

http://www.webmaster...forum48/717.htm has some good run downs on the issue.

http://computer.hows...ks.com/dns5.htm - At the bottom talks about how it took three weeks for a change in DNS propogate through the interenet, and that isn't spiders, that is CUSTOMERS.

All that said, the best method for changing IP Addresses IMHO is simply to keep both up until you are sure that all SE spiders no longer crawl your site, as this is likley to be longer than it takes users to adapt. Depending upon finances and ROI associated with keeping two servers, this could be between 1 and 2 months, depending upon how often you pay, and at what point in the month you change. The desicion you have to make is a variable. If, for example, you have only Teoma still crawling a site after three weeks, and it is only supplying 1% of traffic, but will cost you $20 a month to keep, then you can weigh up the benefit of paying an extra month.

For more information (for the thorough, and let me tell you, if I were doing this, I would be thorough), this search has a lot of information from the tech obsessives over at WMW, who really do know their stuff on issues like this: http://www.google.co...G=Google Search