42 replies to this topic

[NASA]

I'll use other, more proven methods of link pop sculpting if I need to

don't leave me in limbo, spill the beans.

If there is a better, proven , and obviously you think NOT a waste of time method, i'll happily use that instead

[Michael Martinez]

rel="nofollow" does not equate to Do Not Index or Do Not Crawl. All of the engines will still crawl and index pages behind a nofollow. Just that they probably won't rank well if there are no other clean links pointing to them.

Well, people at Google and Microsoft have repeatedly that is not the case at SEO conferences and on various blogs. I haven't seen anyone actually prove that the nofollow links are being followed, although I've seen some interesting case studies (it's almost impossible to "prove" such a claim because the link reporting tools don't report all the links the search engines know about).

For example, Google says "Nofollow" provides a way for webmasters to tell search engines "Don't follow links on this page" or "Don't follow this specific link.".

Yahoo! is a little more ambiguous, as they say Slurp may use a "nofollow" link for discovering content, but the link will not be considered an "approved" link for consideration for ranking of the target page.

Microsoft says: In the event that you need to reference a website that you think might be bad, it is best to use the rel=nofollow parameter inside the anchor tag. This way Live Search doesn't take that link into account when it ranks your website. However, Nathan Buggia says "there are a couple things you could do to ensure that they are not crawling/ indexing your pre-production site ... (3) Use "rel=nofollow" for any public links....".

Last year Matt McGee reported: "After some confusion, Microsoft’s Nathan Buggia said that Live Search doesn’t use nofollowed links to discover new pages."

So I stand by what I wrote. rel='nofollow' tells the search engines "DO NOT CRAWL THIS LINK". It's up to each search engine whether to honor that and only Ask has said they will specifically ignore it.

Your getting functionality/usability mixed up with indexing.

No, I have my ducks in a row on this matter. If you use "rel='nofollow'" you ARE telling the search engines NOT to follow the link. "rel='nofollow'" has nothing to do with "usability" or "functionality".

It's unfortunate that many people, including good forum mods like Randy, are still not clear on these issues but there have been many misunderstandings through the years and as Matt McGee's post from last year points out, a LOT of people have been confused on the matter.

The clearest point you can make is what Randy wrote in followup: use robots.txt to block pages from being indexed. However, if you're attempting to "sculpt PR" you ARE trying to block search engines from crawling a specific page (not just asking them not to pass value). If you don't have any followable links pointing to the page, then the page won't be indexed.

Do NOT count on nofollowed links to get pages into search engine indexes.

[Randy]

It's unfortunate that many people, including good forum mods like Randy, are still not clear on these issues but there have been many misunderstandings through the years and as Matt McGee's post from last year points out, a LOT of people have been confused on the matter.

Or perhaps we'll all have a better understanding once the search engines say what they mean, and more importantly actually do what they say.

I'll run another test to check it Michael. But I ran a series about 4 months ago where I nofollow'd several pages that were only linked to once from a page that is buried so deeply (it's another test page) that nobody else could ever find 'em, let alone link to 'em. In every case the search engines who say they don't follow those links indexed the target nofollow'd pages. I even linked them back to another orphan domain default page and Google Webmaster Tools showed several of those nofollow'd pages as linking to it.

So in other words, don't believe everything they tell you. Apparently sometimes the right hand doesn't know what the left hand is doing. Me thinks they need to have someone from the Crawl team who knows what's actually being done make a statement. Then follow through to make sure everything they say is factual, across the board.

That said, if you use nofollow to tell the search engines you don't trust a page enough to allow link pop to flow it would be quite, quite silly to then expect the same link to be followed for discovery purposes.

[Randy]

don't leave me in limbo, spill the beans.

If there is a better, proven , and obviously you think NOT a waste of time method, i'll happily use that instead

Sure. It's called a well thought out, sensible navigation structure that is both user friendly and spider friendly. No secret there.

[Michael Martinez]

I'll run another test to check it Michael. But I ran a series about 4 months ago where I nofollow'd several pages that were only linked to once from a page that is buried so deeply (it's another test page) that nobody else could ever find 'em, let alone link to 'em. In every case the search engines who say they don't follow those links indexed the target nofollow'd pages. I even linked them back to another orphan domain default page and Google Webmaster Tools showed several of those nofollow'd pages as linking to it.

I believe you'll run as clean a test as possible, Randy, but as far as I am concerned, unless all search engines commit to honoring rel='nofollow' in the same way, any test is really doomed to failure. After all, you might set up a robots.txt directive that blocks 69 crawling search engines -- and SE 70 comes in, follows the nofollow link, and is in turn crawled by other search engines, and you get a cascade effect.

Is that an unlikely scenario? Absolutely. Nonetheless, we cannot be sure we rule out secondary links. Hence, no matter how improbable a leaky link path might seem, it's still possible, and that makes every test suspect.

So in other words, don't believe everything they tell you. Apparently sometimes the right hand doesn't know what the left hand is doing. Me thinks they need to have someone from the Crawl team who knows what's actually being done make a statement. Then follow through to make sure everything they say is factual, across the board.

Well, I'm sure many people agree with you. In this case I suppose it could said that we're not really sure what they are telling us. Yahoo!'s "we honor nofolow but may follow it" position is about as clear as sunshine on a cloudy day.

That said, if you use nofollow to tell the search engines you don't trust a page enough to allow link pop to flow it would be quite, quite silly to then expect the same link to be followed for discovery purposes.

I just think that people who feel they are only preventing a link from passing PageRank and anchor text by attaching rel='nofollow' to it are asking a lot more of the Searchable Web Ecosystem than it is capable of delivering. Google is just one small cog in a huge, complex mechanism.

[NASA]

if you're attempting to "sculpt PR" you ARE trying to block search engines from crawling a specific page

No I'm Not!

Do NOT count on nofollowed links to get pages into search engine indexes.

I don't.

No, I have my ducks in a row on this matter. If you use "rel='nofollow'" you ARE telling the search engines NOT to follow the link. "rel='nofollow'" has nothing to do with "usability" or "functionality".
.

Well these quotes say different

You cannot improve any page's search visibility by hiding other pages.

I'm not, all pages are visible to the search engines.

Thus, one aspect of Torka's "worse" is that you're making it harder for people to find pages on your site.

All pages are accessible to the visitor.

If users cannot find these pages through whatever means of navigation is normal for them

The normal way to find a page is clicking on a hyper link, all hyperlinks work quite normally.

many people also use major search engines in place of navbars and site search to find content on Web sites.

if there was any legal requirement for all your website pages to be in the major SE's , they wouldn't be able to ban you would they! It would be the law.

So trying to "sculpt" PageRank is not only a colossal waste of time, it lowers the quality of the user experience.

how on earth can using a rel='nofollow' attribute on a link , impede, hinder or affect in any way, shape or form a visitor on my site using my links to navigate my site.

Your ducks may be in a row, but I think they're drunk

[Randy]

I believe you'll run as clean a test as possible, Randy, but as far as I am concerned, unless all search engines commit to honoring rel='nofollow' in the same way, any test is really doomed to failure. After all, you might set up a robots.txt directive that blocks 69 crawling search engines -- and SE 70 comes in, follows the nofollow link, and is in turn crawled by other search engines, and you get a cascade effect.

Is that an unlikely scenario? Absolutely. Nonetheless, we cannot be sure we rule out secondary links. Hence, no matter how improbable a leaky link path might seem, it's still possible, and that makes every test suspect.

Spot on observation Michael. And while this is a little bit off topic, others might find it useful.

One of the very simple things I've done for every test designed for search engines over the years that seems to work quite well is I have both the target test page (in this case where the link appears) and also the destination test page (the fake page I upload) record all hits to a flat file or database. This one simple extra(?) step allows me to better control or at least see the bot hits. So not only can I tell when a search engine spider has visited each page, I can then backtrack to see what they've done if anything.

Simple, but it helps to reduce a lot of extraneous noise and to exclude certain spiders that might throw a monkey wrench into the test by excluding them from even seeing the test in the first place.

[oneofthe3lions]

I really have never understood the big hoohaaaa about this unless it is desired to be used to totally keep private pages out of an index. In those cases there are better ways of dong it anyway.

If u dont want to endorse spammy links on your site no follow them, if you dont want unneccassary high pr on privacy notices etc that may lead to poor traffic then no follow them.

pr sculpting is what we have always been doing anyway, 'popularity sculpting' The more important pages have more links to them naturally both hopefully internal and external.

It was brought out to combat the first situation, spammy links or pointing to bad neigbourhoods for various reasons. Then it opened up a large debate because 'people' implied that perhaps a website with product pages 6 layers deep may not be able to flow much pagerank to these deep pages and by no following certain paths they would be able to flow a little more juice to these areas.

Again, it is quite obvious that there are better ways to do this, if a product page is popular and useful point more links to that product page internally and externally. External links in particular don't give a flying f about site architecture so you can get higher pr without doing anything at all on your own site.

I use the no follow on my terms and conditions type notices. No 'value' in search for my target audience. simple as that. (actually it is also plain text so doesnt point to any heirarchy)

So why didnt I just block them via other methods? Well, altho they are not important in a search sense something tells me that is sensible to let 'engines' know that they exist. (hey, it may be one of 200 algorithms nowadays) From all accounts it seems that using this tag 'does' show that they exist so it suits me at this point. I would suggest that it is a very tiny percentage of website owners that use any sort of robots text. Thats scarey stuff for some of us without initial codeing backgrounds.

To me, the only other 'concern' that people may have is probably related to whether they should bother aquiring external links that have a no follow attribute. With the general belief that it will slow down a lot of spam posts because they belive this is true then that can only be a good thing. To us that get links in useful places have always been doing so because of the relevant traffic, absolutely nothing to do with page rank.

So ermm, yes. I just thought it was quite straightforward basically but maybe Im missing something, wouldnt be the first time.

[NASA]

nice post 'oneofthe3lions', you seem to make sense and think as I do regarding some 'superflous' pages, no search value, but no harm in them being in the index or 'known about' by the SE's.

Of course the major form of PR sculpting, is getting IBL's as you so rightly point out!

[Michael Martinez]

Your ducks may be in a row, but I think they're drunk

I think you need time to understand the macro issues involved here. If you think that using "rel='nofollow'" on internal links to important pages on your site won't hurt their visibility, then you have a lot to learn about search engine optimization. You're reducing your crawl for those pages, you're reducing your internal anchor text for those pages, and that reduces their visibility in search results, which in turn reduces the amount of traffic they'll receive.

People who visit your site will still see the links to those pages, but you're sacrificing search visibility for a theoretical (and unproven) boost in value for other pages.

That's the point.

There is no need for you to be insulting just because you don't agree with what I'm saying. I'm offering you advice and feedback that you are free to take or leave, as is everyone else who comments in this thread.

[NASA]

There is no need for you to be insulting just because you don't agree with what I'm saying.

I wasn't and for you to say that is.

I can agree to disagree , accept defeat as well as acknolwledge different opinions, I was having a bit of fun, didn't the abundance of smileys give it away

I don't have to agree with someone here on HRF to respect their opinion, and know i've learnt more on this forum in 6 months, than I ever learnt in 6 years in web design when it comes to SEO!

Come on Michael we're all friends here!

People who visit your site will still see the links to those pages, but you're sacrificing search visibility for a theoretical (and unproven) boost in value for other pages.

Yes I know but that was the whole crux and point of this question & poll.

Have we found the answer yet?

[Jill]

You're reducing your crawl for those pages, you're reducing your internal anchor text for those pages, and that reduces their visibility in search results, which in turn reduces the amount of traffic they'll receive.

I believe NASA said he wasn't looking for search visibility of those pages.

[Michael Martinez]

I blame it on thread drift.