Are you a Google Analytics enthusiast?
More SEO Content
Only Allowing The Homepage To Be Indexed
Posted 17 September 2007 - 10:22 AM
Posted 17 September 2007 - 10:51 AM
With Google, you can try using a wildcard:
Otherwise, you need to explicitly deny access to almost everything:
That will disallow anything beginning with an a, b or c. You would need to add the rest (d to z, numbers, etc.).
WARNING: Please treat both the above with extreme care. Though theoretically accurate, you will need to test. The safest approach is to use the meta robots tag.
Posted 17 September 2007 - 10:55 AM
Maybe the back-end is set up in a way that will let me generate one meta tag for the homepage and a different one for all other pages quite easilly... will have a look.
Is the syntax of robots.txt case sensitive? Would I have to disallow /a and A/ ?
Posted 17 September 2007 - 11:17 AM
robots.txt is case sensitive if your server is (e.g. Apache) and not if your server isn't (e.g. IIS). That's not part of the standard (it's case-sensitive there), but it's my experience of all search engines. Otherwise, it be unworkable on an IIS server.
Posted 30 September 2007 - 08:38 AM
Thanks for the info I'll go down the META tag route.
0 user(s) are reading this topic
0 members, 0 guests, 0 anonymous users