Jump to content

  • Log in with Facebook Log in with Twitter Log In with Google      Sign In   
  • Create Account

Subscribe to HRA Now!

 



Are you a Google Analytics enthusiast?

Share and download Custom Google Analytics Reports, dashboards and advanced segments--for FREE! 

 



 

 www.CustomReportSharing.com 

From the folks who brought you High Rankings!



Photo
- - - - -

We Dont Want Urls To Get Indexed Or Crawled


  • Please log in to reply
4 replies to this topic

#1 incrediblehelp

incrediblehelp

    HR 6

  • Active Members
  • PipPipPipPipPipPip
  • 591 posts
  • Location:Kentucky

Posted 17 March 2009 - 09:14 AM

This is the opposite of what people usually want, lol.

What would be the best way to stop the engines from crawling a specific link on page other than using nofollow? We want to do some tracking on URLs within page that we will want to rank well. These URLs will lead to pages that are indexed and ranking fine. Obviously we donít want to tracking based URLs indexed along with the regular URL. Any ideas from a Google Analytics point of view? I guess we can stick the URL in JavaScript that would hinder them from getting indexed.

The URLs are crawled normally through the sites navigation and we want to offer the URLs on some premium spots on the site that will include tracking.

Any ideas?


#2 Randy

Randy

    Convert Me!

  • Moderator
  • 17,540 posts

Posted 17 March 2009 - 12:15 PM

Let me make sure I understand the question first.

1. You have a page that is already indexed.

2. You're going to be placing some click tracking links pointing to this page, presumably to the same page filename but probably with some sort of query string modifier for your click tracking script to pick up on.

3. You don't want these click tracking urls to be indexed.

Is that about the size of it?

In such a case here's what I'd do, given historical and more recent tools.

First I'd exclude the (hopefully unique) query string in my robots.txt file. For example, if the click tracking was pointing at a page named somepage.php and the query string showed up as ct= my robots.txt entry would look like:

CODE
User-agent: *
Disallow: /somepage.php?ct=*$


Two notes... As long as the query string was truly unique you wouldn't have to set a page in your robots.txt. You could wildcard that too so that all pages with the ct= query string were disallowed. Second, it's been awhile since I tested this so I'm not positive if all search engine spiders these days support * wildcards in url paths. I know Google does, but I'd need to test it again to see if Yahoo, MSN, Ask and all the rest do. I frankly don't remember which do and which don't, other than Google.

Then as a second measure I'd also use the new rel=canonical element in the destination page to list the clean url. That's what it's for after all. Google, Yahoo and MSN have adopted it already. I haven't seen Ask say they do yet, but it'll surprise me if they don't jump on board. That should correct anything that doesn't get caught by the robots.txt wildcard.

#3 incrediblehelp

incrediblehelp

    HR 6

  • Active Members
  • PipPipPipPipPipPip
  • 591 posts
  • Location:Kentucky

Posted 17 March 2009 - 01:26 PM

Yes that is about the size of it Randy.

How about this. Right now we use jump tracking URLs that 301 redirect to the original page. We are using the nofollow attribute and sticking the links in java script:

CODE
<a href="#" rel='nofollow' onclick='window.location="/listing_subscription_clicks/jump/1005/"; return false;'>Name of link</a>


Do you think that would be fine?

#4 Randy

Randy

    Convert Me!

  • Moderator
  • 17,540 posts

Posted 17 March 2009 - 02:07 PM

In theory that should work Jaan. The only downside being that the target page may be losing a bit of link pop from your internal linkage. Not enough to affect rankings most likely.

Either way, I think a double process (my robots.txt combined with rel="canonical" and your nofollow combined with javascript) is probably the best approach. Two chances are always better than one!

#5 incrediblehelp

incrediblehelp

    HR 6

  • Active Members
  • PipPipPipPipPipPip
  • 591 posts
  • Location:Kentucky

Posted 17 March 2009 - 02:49 PM

Well the jump URL is really a brand new URL so I am not sure how the target page could lose link pop as this page is not stealing any. I would think using a variable at the end of the URL would have more of chance to steal link pop away. Remember these links are not needed for SEO value so I am not trying to "save" anything with this move. Just make sure the engines dont crawl the new tracking URL to begin with.




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users

We are now a read-only forum.
 
No new posts or registrations allowed.