21 replies to this topic

[icosa]

A few weeks ago we set up 301 for a lot of several urls on my site.

The aim was to reorganise the content ( added a new menu) and organised pages in appropiate sections order to provide better undestanding on the site structure and navigation for users. And in the same time improve a the name of urls.

In some sections url where tranformed from

www.enerdata.net/enerdatauk/subscriptions/energy.php
to
www.enerdata.net/enerdatauk/knowledge/subscriptions/database/

in other section from this one to:

www.enerdata.net/enerdatauk/eshop/country.php?id=164
to
www.enerdata.net/enerdatauk/knowledge/eshop/united-kingdom-energy-products-information.php


After two weeks old urls wich have the 301 were removed from the SERP.
The problem is last friday, all of them (old urls) came back again to the index of google, and actually we have the old urls and the new urls bieng displaying on the index. The 301 are still alive.

I have not idea why, and I started to analyse the urls under seoconsultants (to verify the 301 code).
For a lot of urls i'm getting "time out".
I don't know why is displaying this time out, and where exactly could came from.

What it should be the next actions to check from my side? Can i get flagged for having both urls too, even if the301 is working? Where the problem came ?
thanks a lot for your feedbacks guys.

Izy

Edited by chrishirst, 17 November 2010 - 06:37 AM.

[Jill]

Either Google is just using an old database (which could very well be) or perhaps your 301's are actually 302's. You might want to check on that.

[icosa]

Either Google is just using an old database (which could very well be) or perhaps your 301's are actually 302's. You might want to check on that.

Hi Jill,
I have checked the 301, they are set well as 301.
I still have the same issue, old and new urls indexed twice since one week.
I have notice in some seo tools (when checking the 301 redirect ) they say "time out". do you think it could be related with the reindexation of old urls?
thank you.
Isabel

[Jill]

I have notice in some seo tools (when checking the 301 redirect ) they say "time out". do you think it could be related with the reindexation of old urls?

Yes.

[qwerty]

I've had something like this happen a few times. On sites that aren't crawled often and/or deeply, Google might discover new URLs by finding links to them when it crawls the home page, but not go back and check old URLs very often. So if new.php is found independently of the 301 you've set up from old.php, and Google doesn't get around to requesting old.php itself for a while, the old URL might remain in the index after the new one has already been added.

[icosa]

I've had something like this happen a few times. On sites that aren't crawled often and/or deeply, Google might discover new URLs by finding links to them when it crawls the home page, but not go back and check old URLs very often. So if new.php is found independently of the 301 you've set up from old.php, and Google doesn't get around to requesting old.php itself for a while, the old URL might remain in the index after the new one has already been added.

This makes quite sense. This site is not deeply visited all weeks, so it could explain the Google behaviour. The thing is quite weird for me is that in the first month , after i set up the 301 the old url where removed from the index. I only had the new 301. And now , since more than two weeks now, they have appeared again.
This is quite incofortable for me getting both indexed. 301 seems working fine.
NOt sure what else i could do. I'm updating some links that remained to the old urls, but appart from this no more ideas..except waiting Google understand i set up 301 .

Well i 'll keep u updated. Hope Gg is going to clean up old urls soon.

[qwerty]

Can you check your server logs and see if, shortly before the old URLs were (temporarily) removed, googlebot had requested them?

In any case, if your old URLs were dropped, even only for a while, that's not the situation I was describing. And I really doubt that since Caffeine went into effect there could be a situation where one data center could fail to share information like that with another for weeks.

Your not blocking access to the old URLs, either with robots.txt or a robots meta tag, are you?

[icosa]

Can you check your server logs and see if, shortly before the old URLs were (temporarily) removed, googlebot had requested them?

In any case, if your old URLs were dropped, even only for a while, that's not the situation I was describing. And I really doubt that since Caffeine went into effect there could be a situation where one data center could fail to share information like that with another for weeks.

Your not blocking access to the old URLs, either with robots.txt or a robots meta tag, are you?

Hi, not robots txt at all,
not meta tag either..
Between the time when the old url were removed and the time gg reindexed the old urls we set up the 404. But nothing to do with 301.. I guess.
Analysing logs to see if ggbot got them../requested them. IF they do, what does it mean?

[qwerty]

It's a bit of a stab in the dark, but I'd like to be able to see the logical sequence of events: Googlebot requests the old URL, the server responds with a 301, Google indexes the new URL and drops the old one. If that's not exactly what happened it may be that the old URLs weren't dropped because of the redirects -- they may not have been dropped at all -- and that might give us a hint about what did happen and how it can be fixed.

[icosa]

Hi, not robots txt at all,
not meta tag either..
Between the time when the old url were removed and the time gg reindexed the old urls we set up the 404. But nothing to do with 301.. I guess.
Analysing logs to see if ggbot got them../requested them. IF they do, what does it mean?

For the logs:
Yes , in fact it accessed at this time. He got righlty 301 for the most of cases, with some exceptions in some pdf . I forgot to update them and they had old links (to the old urls, before the 301). I have updated them this week as i told in a previously. Probably it was enought to reindex again the old url.

[qwerty]

Having links to the old URLs would only potentially result in them getting added back into the index if the redirects are no longer in place.

[Michael Martinez]

Having links to the old URLs would only potentially result in them getting added back into the index if the redirects are no longer in place.

Not necessarily. Google could index the pages with those links but not attempt to follow the links. Remember, there are always lag times involved in these complex processes. If I understand some of the comments above correctly, the URLs probably don't have a lot of internal PageRank -- from which I would infer that their crawl scheduling is not given high priority.

301-redirects are really NOT a solution. They are a band-aid and they buy you time to get outdated linkage updated, as much as you possibly can.

If there are internal links pointing to the old URLs, those links should be updated ASAP.

If there are external links (which can be updated) pointing to the old URLs, those links should be updated ASAP.

[qwerty]

That's true, but under those circumstances Google would probably list the pages in the results of a site: search, but they wouldn't have any information about the content on them (so there wouldn't be any duplicate content to be concerned about), unless they saved the cached information of a URL after they'd dropped it from the index, and I haven't heard of that happening before. Then again, stranger things have happened...

In any case, I definitely agree that you should track down any links to the old URLs and change them as soon as you can.

[icosa]

Not necessarily. Google could index the pages with those links but not attempt to follow the links. Remember, there are always lag times involved in these complex processes. If I understand some of the comments above correctly, the URLs probably don't have a lot of internal PageRank -- from which I would infer that their crawl scheduling is not given high priority.

301-redirects are really NOT a solution. They are a band-aid and they buy you time to get outdated linkage updated, as much as you possibly can.

If there are internal links pointing to the old URLs, those links should be updated ASAP.

If there are external links (which can be updated) pointing to the old URLs, those links should be updated ASAP.

The initial aim is a global reorganisation on the site. For this reason we set up 301,including those sections that have no pr..
all the internal links as far as i know are ok now, but obiously they must be external ones, i can't do a lot on it.
Actually the total number o findexed pages still didn't change...

That's true, but under those circumstances Google would probably list the pages in the results of a site: search, but they wouldn't have any information about the content on them (so there wouldn't be any duplicate content to be concerned about), unless they saved the cached information of a URL after they'd dropped it from the index, and I haven't heard of that happening before. Then again, stranger things have happened...

In any case, I definitely agree that you should track down any links to the old URLs and change them as soon as you can.

I have checked the cache for those old pages still present in SERPS. The date is before i put in place the 301. (almost all are on september).For some other no more cache available. I believe is a good new..
But unfortunatly i 'm still having them on the index.. figures doesn't seem to move a lot . I'll keep you updated on changes, but if you have any other suggestions they are all welcome guys.

[icosa]

The initial aim is a global reorganisation on the site. For this reason we set up 301,including those sections that have no pr..
all the internal links as far as i know are ok now, but obiously they must be external ones, i can't do a lot on it.
Actually the total number o findexed pages still didn't change...
I have checked the cache for those old pages still present in SERPS. The date is before i put in place the 301. (almost all are on september).For some other no more cache available. I believe is a good new..
But unfortunatly i 'm still having them on the index.. figures doesn't seem to move a lot . I'll keep you updated on changes, but if you have any other suggestions they are all welcome guys.

Hi
just to give un update of this. The old site with 301 redirects dissapeared from SERPS(one month later, and traffic and PR tranfert were done until now . SEO traffic was perfect, new urls perfomming really nice.
The bad news is we kept this old domain with 301 and the situation cames up again: Old URLS with 301 are being reindexed now, after more than 6 months with 0 url.
We are really lost, we don't know why GG stars to reindexing again but itselfs !!
I noticed that in GWT the old domain change configuration was desactivated. I have reactivated it but nothing seems to happen.
I believe the best is to close the old domain and set up a robots.txt file no index, and desactivate all the 301, even if there are still a lot of external backlinks.
BR,