2 replies to this topic

[JeremyH]

(Wow, it's been forever since my last post. I just had to update my old picture before making a post!)

I'm developing a wiki using the MediaWiki software.

There are a lot of behind-the-scenes pages that I would like to block from the search engine, so I thought robots.txt.

The problem is, everything is placed in the root folder.

Can I block pages based on how they start:

Disallow: /Special:
Disallow: /Template:
Disallow: /User_talk:

What about pages with different starts, but contain a certain parameter somewhere along the URLs:

Disallow: /index.php?diff=
Disallow: /index.php?oldid=
Disallow: /index.php?printable=
Disallow: /index.php?action=

Feedback is greatly appreciated. Thanks.

[Alan Perkins]

Can I block pages based on how they start:

That is exactly how robots.txt works.

Disallow: /Special:
Disallow: /Template:
Disallow: /User_talk:

Yes, that will work.

What about pages with different starts, but contain a certain parameter somewhere along the URLs:

Disallow: /index.php?diff=
Disallow: /index.php?oldid=
Disallow: /index.php?printable=
Disallow: /index.php?action=

That will work too, but if the parameter may appear anywhere in the URL you may also wish to use wildcards:

CODE

Disallow: /index.php?diff=
Disallow: /index.php?oldid=
Disallow: /index.php?printable=
Disallow: /index.php?action=
Disallow: /*&diff=
Disallow: /*&oldid=
Disallow: /*&printable=
Disallow: /*&action=

[JeremyH]

Thank you very much, exactly what I needed.