Jump to content

  • Log in with Facebook Log in with Twitter Log In with Google      Sign In   
  • Create Account

Subscribe to HRA Now!

 



Are you a Google Analytics enthusiast?

Share and download Custom Google Analytics Reports, dashboards and advanced segments--for FREE! 

 



 

 www.CustomReportSharing.com 

From the folks who brought you High Rankings!



Photo

Robots.txt: Is This Setup Correctly?


  • Please log in to reply
3 replies to this topic

#1 suzstephens

suzstephens

    HR 4

  • Active Members
  • PipPipPipPip
  • 121 posts

Posted 24 February 2010 - 02:48 PM

I'm working on a Wordpress blog that has not yet been indexed. Is the robots.txt file set up correctly? I don't know what this stuff means:

CODE
User-agent: *
Disallow:

Sitemap: http://blog.example.com/sitemap.xml.gz



Thanks for your help!

#2 qwerty

qwerty

    HR 10

  • Moderator
  • 8,549 posts
  • Location:Somerville, MA

Posted 24 February 2010 - 02:55 PM

If your intention is not to block any robots from any pages, then what you have is fine. Specifically, what you've written is that all user-agents are disallowed access to nothing.

#3 suzstephens

suzstephens

    HR 4

  • Active Members
  • PipPipPipPip
  • 121 posts

Posted 24 February 2010 - 03:03 PM

Thanks qwerty!




#4 Jill

Jill

    Recovering SEO

  • Admin
  • 32,775 posts

Posted 24 February 2010 - 03:44 PM

Make sure that your settings in the wordpress admin are to allow spiders to visit and to index. The default is the opposite.




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users

SPAM FREE FORUM!
 
If you are just registering to spam,
don't bother. You will be wasting your
time as your spam will never see the
light of day!