Jump to content

  • Log in with Facebook Log in with Twitter Log In with Google      Sign In   
  • Create Account

Subscribe to HRA Now!

 



Are you a Google Analytics enthusiast?

Share and download Custom Google Analytics Reports, dashboards and advanced segments--for FREE! 

 



 

 www.CustomReportSharing.com 

From the folks who brought you High Rankings!



Photo

Blocking Pages Based On Start Of Url Or If They Contain Parameter


  • Please log in to reply
2 replies to this topic

#1 JeremyH

JeremyH

    HR 5

  • Active Members
  • PipPipPipPipPip
  • 367 posts
  • Location:San Diego

Posted 04 April 2009 - 09:31 AM

(Wow, it's been forever since my last post. I just had to update my old picture before making a post!)

I'm developing a wiki using the MediaWiki software.

There are a lot of behind-the-scenes pages that I would like to block from the search engine, so I thought robots.txt.

The problem is, everything is placed in the root folder.

Can I block pages based on how they start:

Disallow: /Special:
Disallow: /Template:
Disallow: /User_talk:

What about pages with different starts, but contain a certain parameter somewhere along the URLs:

Disallow: /index.php?diff=
Disallow: /index.php?oldid=
Disallow: /index.php?printable=
Disallow: /index.php?action=

Feedback is greatly appreciated. Thanks.

#2 Alan Perkins

Alan Perkins

    Token male admin

  • Admin
  • 1,630 posts
  • Location:UK

Posted 04 April 2009 - 12:52 PM

QUOTE(JeremyH)
Can I block pages based on how they start:


That is exactly how robots.txt works.

QUOTE(JeremyH)
Disallow: /Special:
Disallow: /Template:
Disallow: /User_talk:


Yes, that will work.

QUOTE
What about pages with different starts, but contain a certain parameter somewhere along the URLs:

Disallow: /index.php?diff=
Disallow: /index.php?oldid=
Disallow: /index.php?printable=
Disallow: /index.php?action=


That will work too, but if the parameter may appear anywhere in the URL you may also wish to use wildcards:

CODE
Disallow: /index.php?diff=
Disallow: /index.php?oldid=
Disallow: /index.php?printable=
Disallow: /index.php?action=
Disallow: /*&diff=
Disallow: /*&oldid=
Disallow: /*&printable=
Disallow: /*&action=


#3 JeremyH

JeremyH

    HR 5

  • Active Members
  • PipPipPipPipPip
  • 367 posts
  • Location:San Diego

Posted 04 April 2009 - 08:29 PM

Thank you very much, exactly what I needed.




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users

SPAM FREE FORUM!
 
If you are just registering to spam,
don't bother. You will be wasting your
time as your spam will never see the
light of day!