2 replies to this topic

[ttw]

Hi: I have a client that has customer portals that look like this:

example1.clientsite.com
example2.clientsite.com
example3.clientsiste.com

Each sub domain could have many pages with login screens, T&Cs, Forgot your password, pages.

The client site has about 500 URLs out of a total of over 6,000 URLs in Google when we do a < site: >.

We see many URLs that look like: example3.clientsite.com/a/234j3k4

We still want visitors to have access to these pages - just not the search engines.

One idea our contractor proposed was to

• use a canonical tag for every URL that looks like this: example3.clientsite.com/a/234j3k4
• include a Robots NOINDEX META tag:<META NAME="ROBOTS" CONTENT="NOINDEX">

While this seems like a time-intensive way to address this problem for so many URLs. Our contractor is saying that this will alert Google to remove the URL from their index. see:

"" When we see the noindex meta tag on a page, Google will completely drop the page from our search results, even if other pages link to it."


Wouldn't it be better to use block all subdirectories using robots.txt?

Would this technique also signal Google to remove these URLs from the index -- but would it take longer?

Thanks.

Edited by ttw, 05 November 2012 - 12:33 PM.

[Jill]

I agree that having noindex on the pages is what is necessary. I don't think you need the rel=canonical, however.

[Alan Perkins]

Wouldn't it be better to use block all subdirectories using robots.txt?

 

Personally, yes, I would use robots.txt if you don't want any of the pages on those subdomains to be indexed.

 

In fact, if customers are having to log in to view those pages, then an even better way might be to issue a HTTP 401 response that forces them to log in there and then.  It that way, Google won't index them.