Jump to content

  • Log in with Facebook Log in with Twitter Log In with Google      Sign In   
  • Create Account

Subscribe to HRA Now!


Are you a Google Analytics enthusiast?

Share and download Custom Google Analytics Reports, dashboards and advanced segments--for FREE! 




From the folks who brought you High Rankings!

- - - - -

What's The Best Way To Block Search Engines From Subdomains With L

  • Please log in to reply
2 replies to this topic

#1 ttw


    HR 5

  • Active Members
  • PipPipPipPipPip
  • 390 posts
  • Location:San Mateo, California

Posted 05 November 2012 - 12:25 PM

Hi: I have a client that has customer portals that look like this:


Each sub domain could have many pages with login screens, T&Cs, Forgot your password, pages.

The client site has about 500 URLs out of a total of over 6,000 URLs in Google when we do a < site: >.

We see many URLs that look like: example3.clientsite.com/a/234j3k4

We still want visitors to have access to these pages - just not the search engines.

One idea our contractor proposed was to
  • use a canonical tag for every URL that looks like this: example3.clientsite.com/a/234j3k4
While this seems like a time-intensive way to address this problem for so many URLs. Our contractor is saying that this will alert Google to remove the URL from their index. see:

"" When we see the noindex meta tag on a page, Google will completely drop the page from our search results, even if other pages link to it."

Wouldn't it be better to use block all subdirectories using robots.txt?

Would this technique also signal Google to remove these URLs from the index -- but would it take longer?


Edited by ttw, 05 November 2012 - 12:33 PM.

#2 Jill


    Recovering SEO

  • Admin
  • 33,225 posts

Posted 05 November 2012 - 12:51 PM

I agree that having noindex on the pages is what is necessary. I don't think you need the rel=canonical, however.

#3 Alan Perkins

Alan Perkins

    Token male admin

  • Admin
  • 1,648 posts
  • Location:UK

Posted 06 December 2012 - 06:48 AM

Wouldn't it be better to use block all subdirectories using robots.txt?


Personally, yes, I would use robots.txt if you don't want any of the pages on those subdomains to be indexed.


In fact, if customers are having to log in to view those pages, then an even better way might be to issue a HTTP 401 response that forces them to log in there and then.  It that way, Google won't index them.

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users

If you are just registering to spam,
don't bother. You will be wasting your
time as your spam will never see the
light of day!