30 replies to this topic

[Scottie]

FWIW- I do not recommend this, although I don't see anything wrong with it either.

I've been having a conversation with someone who wanted to block off unnecessary pages in their site from being indexed/receiving PR such as shipping, privacy policy, etc.

They thought that using a robots.txt exclusion or a robots meta on the page would do it. I agree that will stop the page from being indexed, but not stop PR from passing to it.

When PR "votes" are tallied from a page of links, whether or not the links are valid isn't calulated when dividing up the percentage of the "vote" that is assigned to each outbound links. Therefore links to unindexable or broken pages are still reducing the value of each link on the page.

IMO, the only way to "prevent PR" from leaking to other pages is to mask them with javascript or some other technology that makes them invisible to the spider so they aren't used in the calculations for that page.

IMO- it's more trouble than it is worth and I believe in optimizing all pages instead of hiding some, but if you don't want spiders accessing or "counting" certain pages, that is your perogative- nothing wrong with it.

Opinions?

[qwerty]

The idea is to prevent PR from leaking to internal pages? I don't see the point. Assuming the shipping and privacy policy pages link to the home page, the PR that's being distributed to them is being distributed back to the rest of the site as well. If anything's leaking it's negligible.

[Scottie]

Well, that's my point too, Bob. But if someone wanted to do that, I don't see a problem with it. There are people who believe that the PR is weakened by spreading it over additional pages. I believe the more pages, the better personally.

If you wanted to test the theory, hiding the links with unspiderable javascript is the only way I see to do it.

[Grumpus]

I see what you're getting at Scottie. The link appears on a page that is crawled, but it goes to a page that's banned via robots.txt. That link does still want to pass the PR, but there's no page for it to pass to, so it's wasted.

The problem here comes in the notion that the desire here is to manpulate PR. I'm not saying that anything has been done by Google, but if I worked there and we discovered that this type of thing was being used to manipulate PR, I can think of several ways to programatically detect this type of thing.

I always tend to recomend that if your intent is to manipulate PR, then you are walking a dangerous line. (Don't get me wrong, I'm not talking about things like "getting links to increase PR" - that's not manipulation. Manipulation is to do something artificial to affect it in some way).

The only way you can make some links "count" and others "not count" would be to mix javascript with regular linking. This will present user problems be it from disabled javascript or be it statusbar surfers (I'm a statusbar surfer. I never click a link that I can't see what it's going to do.) or be it something else. There are ways around everything, but really, for each link that's getting lost, it only takes one more inbound link to counter that loss.

I think your proposal is low to medium risk with no better than minimal gain potential. The math just doesn't work for me.

G.

[Scottie]

Haven't we determined that some javascript links are being followed anyway, Stock?

So, even the way I have it laid out might not work.

[Ruud]

I haven't heard anything yet about SE's following javascript links so to me if you really want to prevent PR from 'leaking' then you can link with an onclick event. Heck, you can even have the HREF go to a page you *want* PR to go to while the onclick event does lead to the page you intended. Of course those with javascript turned off will end up at a wrong page, lol.

Ruud

[powerofeyes]

There are people who believe that the PR is weakened by spreading it over additional pages.

Didnt we discuss this before scottie, PageRank is only passed to other pages, the page itself possessing the PageRank wont loose any PageRank value,
So why that person have to worry about, there is nothing called PageRank leakage or a page loosing PageRank because it has a link on it,

[Scottie]

This is a little different, Vijay. It's not about losing PR, but about trying to concentrate the PR on fewer relevant pages instead of spreading it out to pages that the site owner isn't worried about having indexed or anyone searching for it.

Personally, I don't know that more pages lessens the "strength" of PR, I tend to think it increases with each additional page. But I don't know that for sure. And the person I was talking to wanted to test the theory by making the non-essential pages invisible to SE spiders.

The reason I posted was that in talking to other people, some thought that the robots exclusion would do the job alone, but I see that hurting more than helping. As I stated above, I believe the link gets calculated, even if it goes to a page that can't be indexed.

[Jill]

I can't imagine there ever being a reason why you'd not want to pass PR to all the pages of your site. If you want to exclude them from the spiders so that they'll go about their business of getting the important pages, that I can see, but not for PR purposes.

If you want to control your PR, simply link all the important pages from the home page, and don't link those ones that aren't so important from there, make them a few clicks away.

You get a higher PR on pages that are the fewer clicks away from the home page, so you can use that to your advantage (if you care about such things).

Jill

[Jill]

Personally, I don't know that more pages lessens the "strength" of PR, I tend to think it increases with each additional page.

Since Google shows links from your own site in the backlinks check, then I'd have to agree that each page helps strenghten the PR of the overall site.

Backlinks from my site show tons of my own links, and I would imagine they all help! I would never want to exclude them!

Jill

[Ron Carnell]

Personally, I don't know that more pages lessens the "strength" of PR, I tend to think it increases with each additional page.

Absolutely!

If we had three billion pages on the Internet with NO links at all, every page would have an intrinsic PR of unity (which isn't the same as PR 1 on the toolbar, of course). That value is built into every page created, a gift from Google simply for existing. Were that not true, if new pages started out with 0 PR, the multiplications in the PR formula would result in that PR 0 being applied to the whole Internet. The Stanford formulas only work if all pages start out with a non-zero value.

So, every new page you create on your site automatically increases the total PR of your web site. That doesn't mean a whole lot by itself, because PR isn't assigned to a site, but to a page.

Let's say I have a one-page site, and that one page has a PR of 4. Hey, it could happen! Now I create a new page, with a built-in PR of unity (not the same as TBPR 1, remember). I link to the new page from the home page, and the new page gets the benefits of that "vote," to the tune of (we're guessing) unity plus .85 * 4. That should give the new page a PR of high 3 or low 4. If the new page then links back to the home page, that too counts as an incoming link, or vote. So the home page now is recalculated, getting say, .85 * 3, plus it's original 4. That might actually be enough to push it to a low 5.

The point is, by creating a new page and linking it back to the home page, you almost automatically increase the PR of the home page. I used a pretty far-fetched example, because creating just one new page probably won't show any visible change on the toolbar. But it could, and a few thousand new pages almost certainly WILL. Barring human interference (ala SearchKing), there is no such thing as a link that doesn't count. And that's a direct result of every new page starting with a PR equal to unity.

I think there are very valid reasons to use robots.txt to block pages from being indexed. I have about 18,000 links, for example, that lead to forms that are different enough to be indexed but similar enough to be a waste of everyone's time. They will never be found in a targeted search, so I block them in robots.txt in hopes Google will deep crawl more important pages instead. I do that, though, at the COST of Page Rank, not to preserve it. Every page blocked from the index lowers the total PR of the site, thereby lowering the potential PR that can be passed to other pages.

IMO, no one should ever worry about leaking potential PR. Just create more pages of good content to compensate.

[OldWelshGuy]

I have run the figures on this a few times lately, and there is a way that people are manipulating their PR. I do see some merit in it, but on the other hand I think that if you are sacrificing usability, or ANYTHING for that matter, then what you are doing is wrong.

It does seem like Google is trying to dump the PR leeches, over and above the spammers. I know this is a bold thing for me to say, but the more I think about it, the more I seem to feel it is right.

PR leeches get as many links IN to their site as they can, then they bottle it within that site, directing it to the pages they want it to go to, and away from the pages they do not want it to go. Pages like forms, contact us etc they feel do not warrant any 'votes' as their purpose in life is to collect and inform, not rank.

This would explain why the directories, and other free linking sites suddenly flew to the top, as Google tried to dump these leeches, directories and free linking sites fall into the style of web that Google sees as the 'right' way, the Google way.

As we all know, the whole Google principle is built around networks, networks, of sites, groups of well informed people voting for each other. This is still at their heart as can be seen from Orkut something created by its employees as a project, and then being accepted as 'worthy' of being associated with the Google brand.

SO if every page links TO every page, then all pages are equal. But if you 'daisy chain' non important pages, then in theory if every one of those pages has only two links, one to the next page in the chain, and one to the home page, then the home page is milking PR.

By using non spiderable links FROM the home page (image maps flash etc) but using spiderable links BACK to the home page (or whatever chosen page) then you can, (as Ron pointed out) have 'boosted' internal PR links.

Personally I can see that this is walking a tightrope, and is borderline spamming, but i do agree that having a form page with the same PR as your home page is equally not the way it should be.

[Grumpus]

Haven't we determined that some javascript links are being followed anyway, Stock?

I haven't seen any Javascript pages getting followed (though that doesn't mean it's not happening). I think what you're refering to is the occasional credit links are getting over at cre8asite even though they go through the php "link blocker" script. This seems to be more of a spider belch, not Google deciding it wants to credit all links. At first I was concerned about the implications, but as I've looked into it more, it's just a hiccup, I think.

Note for others - at cre8asite, we put all outgoing links through a redirect page that is banned to robots. The reason we do this is not to hoarde PR, but rather, to avoid our forums being linked to those dreaded "bad neighborhoods" that'll get you penalized. Here, Jill and crew edit out URLs that look to be potentially troublesome which serves the same purpose. We're just lazier over at cre8asite than the high rankings folks.

G.

[awall19]

Sometimes you would not want to parse pagerank. Of course you want credit for your own content. Occassionally though there are scenarios where linking to pages can hurt you. In that scenario I would use java.

Why would I link to pages that could hurt me?

Lets say I had a porn directory and was an avid porn fan.
My black hat seo directory.

I sometimes link to bad stuff as an example of what not to do.

Many sites also "sell out" by throwing some "buy phentamine" links on their sites. If some of their content is good, but I did not want to support "buy phentamine" I would link to them with java.

[Jill]

Here, Jill and crew edit out URLs that look to be potentially troublesome which serves the same purpose.

For the record, it's not so much for the bad neighborhood effect as it is for the drive-by'ers who just want to get a link or an ad.

But regarding the topic at hand, I'm glad to see Ron's post which seems to validate exactly what I (and I think Scottie) were thinking intuitively on this matter. I don't really get the math involved, but what Ron said simply makes sense.

Jill