4 replies to this topic

[Steveharro]

Hi everyone

How do I go about blocking bots from a section of my site.I want to block them from a directory that has hundreds of html pages that don't go anywere they are to copy and paste code for scripts and then to be closed.I have read the google page on the subject but don't fully understand it.

[Gatorhardware]

You can create a robots.txt file. Here is some info on it.

http://www.robotstxt.../wc/robots.html

[Steveharro]

Yeah thanks for that I've read it all but still don't get it.I must be a bit thick. All I want to do is block off one directory called scripts.Can someone please give me a hand.

[SearchRank]

Simply create a text file and call it "robots" which will then read as "robots.txt.

Include the following it it:


User-agent: *

Disallow: /scripts/


Upload it in your root directory. That's all there is to it.

[Steveharro]

Thanks for that is it that simple I thought there was more to it than that.