Jump to content

  • Log in with Facebook Log in with Twitter Log In with Google      Sign In   
  • Create Account

Subscribe to HRA Now!

 



Are you a Google Analytics enthusiast?

Share and download Custom Google Analytics Reports, dashboards and advanced segments--for FREE! 

 



 

 www.CustomReportSharing.com 

From the folks who brought you High Rankings!



Photo
- - - - -

What's The Best Way To Block Search Engines From Subdomains With L


  • Please log in to reply
2 replies to this topic

#1 ttw

ttw

    HR 5

  • Active Members
  • PipPipPipPipPip
  • 366 posts
  • Location:San Mateo, California

Posted 05 November 2012 - 12:25 PM

Hi: I have a client that has customer portals that look like this:

example1.clientsite.com
example2.clientsite.com
example3.clientsiste.com

Each sub domain could have many pages with login screens, T&Cs, Forgot your password, pages.

The client site has about 500 URLs out of a total of over 6,000 URLs in Google when we do a < site: >.

We see many URLs that look like: example3.clientsite.com/a/234j3k4

We still want visitors to have access to these pages - just not the search engines.

One idea our contractor proposed was to
  • use a canonical tag for every URL that looks like this: example3.clientsite.com/a/234j3k4
  • include a Robots NOINDEX META tag:<META NAME="ROBOTS" CONTENT="NOINDEX">
While this seems like a time-intensive way to address this problem for so many URLs. Our contractor is saying that this will alert Google to remove the URL from their index. see:

"" When we see the noindex meta tag on a page, Google will completely drop the page from our search results, even if other pages link to it."


Wouldn't it be better to use block all subdirectories using robots.txt?

Would this technique also signal Google to remove these URLs from the index -- but would it take longer?

Thanks.

Edited by ttw, 05 November 2012 - 12:33 PM.


#2 Jill

Jill

    Recovering SEO

  • Admin
  • 32,865 posts

Posted 05 November 2012 - 12:51 PM

I agree that having noindex on the pages is what is necessary. I don't think you need the rel=canonical, however.

#3 Alan Perkins

Alan Perkins

    Token male admin

  • Admin
  • 1,639 posts
  • Location:UK

Posted 06 December 2012 - 06:48 AM

Wouldn't it be better to use block all subdirectories using robots.txt?

 

Personally, yes, I would use robots.txt if you don't want any of the pages on those subdomains to be indexed.

 

In fact, if customers are having to log in to view those pages, then an even better way might be to issue a HTTP 401 response that forces them to log in there and then.  It that way, Google won't index them.






0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users

SPAM FREE FORUM!
 
If you are just registering to spam,
don't bother. You will be wasting your
time as your spam will never see the
light of day!