Are you a Google Analytics enthusiast?
More SEO Content
Posted 08 December 2004 - 01:28 PM
A question about the robots.txt file.
I have a dynamic PHP page which is a template for hundreds of pages, but I don't want any SE spidering these pages due to potential duplicate page penalty, the url example is www.mydomain.com/thefile.php?id=2323 so the 2323 will be a different ID for each page.
In the robots.txt file do I just put in the URL www.mydomain.com/thefile.php or do I have to list every single id?
Posted 08 December 2004 - 02:15 PM
will work and
should also be ok
If these are affiliate/link IDs you could use php to strip the querystrings and redirect the URLs back into the page, which should (in theory at least) transfer the link pop without any dup risk.
Posted 09 December 2004 - 09:56 PM
I have a file www.mydomain.com/property.php?home=232322
I have altered the mod-rewrite file to make the file www.mydomain.com/232333
I don't want Google to spider both of the URL's so if I put in the robot.txt file
Will this work?
I have a sitemap which I will list all of the properties using the www.mydomain.com/232333 domain Will it still spider this page okay if I stop the bots spidering www.mydomain.com/property.php?home=232322
Posted 10 December 2004 - 08:01 AM
The answer to the second question is Yes. Disallowing the actual name and providing the mod-rewrite version on your site map will accomplish the task.
Posted 17 December 2004 - 01:54 PM
Will this not allow anyone to be able to open that file?
Posted 17 December 2004 - 02:34 PM
0 user(s) are reading this topic
0 members, 0 guests, 0 anonymous users