2 replies to this topic

[delpreston47]

Hi again guys,


It's my first day on this forum and after reading a bunch of the previous threads, I had noticed that everyone is talking about this meta tag or file called "robot.txt". I've never heard of it before. I thought that the keyword meta tag and description meta tag were enough to do the trick but as in my case my site gets little to no traffic and is a very helpful site for people wanting to learn how to play guitar. Can someone give me a little short explanation of robot.txt, and or check out my source code for my main page and tell me what they think is wrong?

I've also been kind of "Hustling" (going to other guitar websites and asking for link exchanges) to boost my traffic on my website. I notice that very few people get back to you and it can be frustrating. Thanks everyone I really appreciate the help!

:-)

cheers,

Justin

Edited by Jill, 12 September 2006 - 12:59 PM.

[dburdon]

Justin,

the robots.txt file instructs search robots (spiders). It can prevent spiders from crawling your site, any particular section or any individual page. If its not there the spider will crawl anyway.

Presuming you want your site to be crawled I would suggest using the following tag:

<meta name="robots" content="index,follow">

[Jill]

delpreston47, please see the Tips for Newbies articles which discuss the meta tags.

They are not the key to high rankings, let's put it that way!

(And neither is the robots.txt file. See our robots.txt file forum for more on that one.)

Oh, and this code won't do anything either:

<meta name="robots" content="index,follow">

The robots follow and index by default.