Are you a Google Analytics enthusiast?
More SEO Content
Permanent Redirect For Robots.txt?
Posted 05 March 2005 - 07:10 PM
I think this is because I consolidated all of my domains into one site, and the old names redirect you to a specific folder on my main site.
So my question is can I put a 301 redirect back to my top level robots.txt file for any request for a robots.txt file in one of subdirectories? Does it hurt, help, or have no affect either way.
Alternatively, what if I put a robots.txt file in my subdirectories. Any affect at all?
All thoughts are welcome.
Posted 05 March 2005 - 07:34 PM
Posted 05 March 2005 - 07:41 PM
Like Qwerty said, it should correct itself over time. Once they realize that the old domains really aren't there any longer.
But in the meantime, yeah you can do what you want. To make sure I'm understanding you correctly Richard, all of these robots.txt files are in sub-directories, right?
If so, on a *nix system this single line in the .htaccess --placed at the root level only-- should do the trick.
Make sure to change out the YOURDOMAIN with your actual domain name.
What the redirect match is basically saying is, I want to look for anything at least one level below the root level (or slash, wildcard, slash) that is named robots.txt. Then forward those requests back to the robots.txt file at the root level.
Posted 09 March 2005 - 02:46 PM
REDIRECT TO: siteb.com/
REDIRECT TO: siteb.com/folder
Meaning that the files and folders that the person is looking for gets passed along with the redirect.
So if you have the redirect go into a subdirectory like this:
Then if it's looking for the robots text at sitea.com, it will get redirected to siteb.com/sitea/robots.txt like your having it do now, even through it's not the root.
I see no problem with having the robots.txt in your subdirectories. Give them what they want.
Though, I would think that over time, your old pages will be dropped by the Search Engines as your new content is being picked up, so this whole thing may be a moot point.
0 user(s) are reading this topic
0 members, 0 guests, 0 anonymous users