Are you a Google Analytics enthusiast?
More SEO Content
Posted 14 April 2004 - 12:54 PM
A little while ago I created a robots.txt file and put it at the root of my website to stop some webpages being indexed. However, it hasn't stopped the pages from being indexed. Can you confirm that I have the correct format please:
Many thanks <--This is not in the robots.txt :o)
Posted 14 April 2004 - 01:00 PM
Posted 14 April 2004 - 01:06 PM
Posted 14 April 2004 - 01:13 PM
User-agent: *like you have it in your post? If that's the case, you need to remove it. Quoting from http://www.robotstxt...sion-admin.html :
...you may not have blank lines in a record, as they are used to delimit multiple records.
Posted 14 April 2004 - 01:38 PM
I did have a blank line in there. That explains it.
What will happen now? Will google remove those pages the next time it reads my robots.txt?
Posted 14 April 2004 - 01:49 PM
I didn't find anything similar in Yahoo, except a way to remove a site from their directory, and you obviously don't want that.
0 user(s) are reading this topic
0 members, 0 guests, 0 anonymous users