72 replies to this topic

[Alan Perkins]

OK something else, noone seems to address: no-followed links are not even used for discovery. Can anyone tell me this referred to ON SITE no-followed links only ?

No, it refers to all links.

There are lots of other means of discovery though. It's no surprise that a page, that you thought wouldn't be found, was found. That kind of thing happens all the time.

[Randy]

But again, what about google? Is there any reason why the NoFollow should not be used to prevent indexation of pages with non-content effecting parameters?

To my knowledge Google doesn't yet support any such tool in their Webmaster Central area yet. There are other ways to basically accomplish the same thing, nofollow being one of them. With the caveat that the nofollow destination page needs to not be linked to from anywhere without the nofollow.

Since Google supports some non-standard (or Extended Standards is the way they word it) robots.txt instructions you can in fact exclude these non-content query string pages. But you need to understand that excluding them via robots.txt does not attempt to recapture PageRank.

The concept makes use of Googlebot's recognition of both pattern matching and wildcard characters in robots.txt files. Which is something that's not in the robots.txt standard. The short version goes something like this...

Let's say you had a series of URLs that used a query string that was being used to identify what browser someone was using. The query variable name was something like browser= and could have one of several values.

If you wanted to exclude these URLs from Google's index because of duplication you could use something like the following in your robots.txt file to get them all in one fell swoop.

CODE

User-agent: Googlebot
Disallow: /*?*browser=*

Make sense? It's excluding any url that contains any page filename, followed by a ? mark, followed by the browser= variable name. Wildcarding a bunch of stuff inbetween each portion. But it doesn't block any page that doesn't include the browser= variable.

An important note being, I've never resorted to this sort of thing myself because I manage it in other --some say more complicated-- ways when the need arises. Which isn't very often. Sadly there's just not an easy way to manage such things. No matter which method one choose, there are potential downsides, potential false positives and potential false negatives that one must account for.

If you decide to employ such a robots.txt query string/pattern matching method I would highly encourage you to use the robots.txt validator Google makes available via their Webmaster area to make sure it's not going to cause them any issues.

[Alan Perkins]

Is there any reason why the NoFollow should not be used to prevent indexation of pages with non-content effecting parameters?

The main reason is that nofollow (either in a link attribute or in a robots meta tag) is a really, really poor way to prevent indexing. noindex and robots.txt are better ways, and Randy's solution above is a good one (although I always try to avoid using Google's non-standard extensions to robots.txt when I can).

Regarding dropping query parameters from URLs, it's a great idea but Matt Cutts ran a poll where this was one one of the options and very few people expressed an interest in it. So maybe it won't be top of Google's priority list.

[Randy]

Too true Alan.

Unfortunately I think Matt's poll is flawed unless it's dynamically setting the order in which items appear. I actually got a good laugh from that one the first time I saw it since it says something quite intriguing about Matt's Users. Or was I the only one who noticed that the Results exactly match the order of the list during the Voting stage? It's rare to see such an exact match.

Makes me wonder what it show if Matt inverted the list and asked the same question again.

Yeah, yeah... I know. I've just been testing too much stuff lately to notice these tiny things first.

[Alan Perkins]

Well spotted!

You've gotta think that's the software ordering it ... very odd either way, though.

[nethy]

Thing is I am not nessecarily trying to prevent indexing. I am trying to prevent PR from being directed at both 'pages' if a link is not nofolollowed its not really a very big 'loss' of page rank.

Taking it a step back: This seems silly. Google's job is to decide what pages get page rank & how much. Sites are more complex these days then they were back in the day. They don't (rightly) trust webmasters to tell them how to treat a site. That means they need to come up with a way themselves to figure out what is a unique page.

[DanThies]

Hey folks - Bob suggested that I post these links - a couple posts from my blog intended to provide some clarity and practical examples:

First post, explaining what nofollow is and isn't, and when you might want to use it (or not):
http://www.seofastst...l-nofollow-help

Second post, with a couple diagrams and cases:
http://www.seofastst...follow-examples

[Kestrel]

Hi all,

I understand the concern with flagging a website. Most SEO's these days try and avoid the 'over optimised' look by varying titles from header 1's and chucking a few non-relevant keywords in our headers and anchor from time to time.

But if a well SEO'd site was something to worry about, why don't all the highly optimised websites owned by SEO companies get flagged?

I agree that using the nofollow attribute is just another way to direct PR, as has always been done with the robots protocol.

I guess the question for most folk is, is the rel="nofollow" technique for achieving this same result white hat?

As it has been endorsed by the man himself then clearly it is.

However, i still feel we shouldn't depend wholey on it. Perhaps seek to use the usual internal linking methods and maybe just add this technique to improve performance a tad with Google.

Yahoo and MSN traffic may be minimal in comparison, but it's still traffic. And who knows, maybe one day Google may not be the giant it is now.

Lastly, I will defo be considering using the rel=nofollow on links with the anchor Home and using relevant anchor in the sitemap, footer links and in-content links in-place.

*edit*

You know, if the keyword in the anchor isn't relevant, one can always use the href title attribute...

K

Edited by Kestrel, 14 September 2007 - 03:59 AM.

[Jill]

But if a well SEO'd site was something to worry about, why don't all the highly optimised websites owned by SEO companies get flagged?

What makes you think they're not?

[Kestrel]

What makes you think they're not?

Many probably are, but many aren't. Just saying that if being open about SEO'ing a website could get it penalised then surely these websites would be the first to cop it.

They'd be nothing left in the serps bar SEO articles and tutorials, probably what Google would prefer.

I'm gonna zip it now in case they're listening

[Jimmy Dunne]

What makes you think they're not?

Now the SE's are flagging sites that bother to optimize linking and title tags? Ummm, that would be a big bucket of sites since you could learn basic onpage SEO from any $19.99 ebook these days.

[Webmaster T]

No, the meaning it had - note, had - was don't trust this link, not don't follow this link. Then it became "This link is paid for". Now it's "Manipulate your PageRank here". That's what I mean by bastardised. Maybe, to Google, it's been like that all along and it has just been a slow process of discovery for the rest of us. But the way Google treats nofollow is not the way it was designed to be used, and is not the way that other engines do (or may).

Allan you have posted many times on how the Robots tag and Robots.txt protocols are wonky. By wonky I mean they don;t seem to work the same on all engines. I would say when it comes to this the engines have a dismal record of managing these protocols and tags. The Google waffle seems to indicate this will be no different than the others, namely, each will do with it what they please and damn those who have to interpret and implement it. I won't use it until the SE's put it in the hands of the rightful governing body since it is just a bastard <href> attribute that AFAIK does not pass an HTML lint test. IMO, paid links were around when they wrote the algo, unfortunately they left a hole in it you could drive a Mac truck through. Why don't they just weight the links appropriately and then paid links are moot or hardly worth buying!

[Scottie]

Hear hear!!

Good to see ya, Terry.