3 replies to this topic

[Andy_Seo]

Hello all.

I have a website that has a number of canonicalization issues - one of them being a .index.php and index.php?noflash indexed.

Now it is exactly the same content and my question is what affect will the restriction of /index.php?noflash have on the .index.php of the site? Will the robots still be able to crawl the root directory?

Cheers.

[Randy]

Google would see the distinction. I believe Yahoo would too. I've honestly not paid attention to MSN or any of the rest, but would assume they can handle the dynamic url in robots.txt.

In any case if the engine in question has your site indexed with just the base url with no index.php in there at all they should all see the difference. Watch it after you change it in different engines to make sure. It's been awhile since I've tested this sort of thing. That way if you need to change something for one engine or another you'll know it right away.

[Jill]

Now it is exactly the same content and my question is what affect will the restriction of /index.php?noflash have on the .index.php of the site? Will the robots still be able to crawl the root directory?

Excluding /index.php?noflash would have no effect on index.php. The search engines will only exclude the one URl you have listed there. That's exactly how you should d o it.

[Andy_Seo]

Many thanks for you input guys. Your posts have confirmed my initial thoughts.