Are you a Google Analytics enthusiast?
More SEO Content
Need A Robot.txt File?
Posted 09 June 2004 - 11:50 PM
Posted 10 June 2004 - 12:16 AM
But its so simple to add one, why not have it?
Posted 10 June 2004 - 12:36 AM
but you can basically create a brand new text file and put *only* the following text in:
Another good one would be:
Which prevents the spiders from going into those directories but allows everywhere else
The only change here is the slash / but it will turn an "allow everything" into an "allow nothing" - so be very careful.
Once you have created this file call it robots.txt and upload it so that you could find it here:
Make sure that you have set permissions so that it's readable, and you are done.
0 user(s) are reading this topic
0 members, 0 guests, 0 anonymous users