Jump to content

  • Log in with Facebook Log in with Twitter Log In with Google      Sign In   
  • Create Account

Subscribe to HRA Now!

 



Are you a Google Analytics enthusiast?

Share and download Custom Google Analytics Reports, dashboards and advanced segments--for FREE! 

 



 

 www.CustomReportSharing.com 

From the folks who brought you High Rankings!



Photo

Robots.txt - Disallow: /*xyz


  • Please log in to reply
2 replies to this topic

#1 djingo

djingo

    HR 1

  • Members
  • Pip
  • 7 posts

Posted 18 October 2009 - 05:44 PM

Does

Disallow: /*xyz

- in a Robots.txt mean that all urls containing xyz will be ignored?

In www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449 it says

QUOTE
To block access to all URLs that include a question mark (?) (more specifically, any URL that begins with your domain name, followed by any string, followed by a question mark, followed by any string):
User-agent: Googlebot
Disallow: /*?


I am a little in doubt if this only goes for "?" or any string following the star.

#2 Randy

Randy

    Convert Me!

  • Moderator
  • 17,540 posts

Posted 18 October 2009 - 10:08 PM

QUOTE
Disallow: /*xyz

- in a Robots.txt mean that all urls containing xyz will be ignored?


In Google's implementation, yes. All urls containing the text string "xyz" would be excluded from spidering. They allow use of a * as a wildcard character.

I'm putting this into a Google perspective because technically I don't believe the robots.txt specs had any such wildcard character. Most if not all search engine spiders support it though.

Moving this one to the robots.txt area of the forums.

#3 djingo

djingo

    HR 1

  • Members
  • Pip
  • 7 posts

Posted 19 October 2009 - 05:11 PM

Ok, thanks!




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users

SPAM FREE FORUM!
 
If you are just registering to spam,
don't bother. You will be wasting your
time as your spam will never see the
light of day!