65 replies to this topic

[sweepthelegnate]

oooh you go away for a day, come back and look what you find...

great stuff, thanks grumpus.

do you have a link to any of the orginal docs? I'd like to read them.

[Irony]

Yes, great stuff indeed, thanks a lot Grumpus.

A couple of notes:

It WILL help to have links from a links page that has an identifiable topic and one where the topic has been determined by Google, but it's hard to know which ones have.

Why do you think it's so hard? I believe, if a link page is dedicated to one topic (strictly), Google shouldn't have trouble identifying it. Link pages where topics are mixed and links are placed for the sake of links only, should be of no use. Right?

Outbound links are now critical to your success, as well. I can lie in my content, so a SE can't place all its ranking weight on content because of that. I can't control who links to me nor can I always even understand WHY someone has linked to me, so using inbound links to my page isn't perfect. But, even though it would seem to be the contrary - it's my outbound links that are the hardest to spoof and lie about. Let's look at it...

People who still think hard how to hide their outbound links from the SEs and "converse PR", should all read this BTW, does it mean those who are already abusing the rel="nofollow" thing to hoard PR, are likely to suffer from this strategy?

[Grumpus]

Sorry Debra, didn't mean to seem short.

do you have a link to any of the orginal docs? I'd like to read them.

Here's the patent for Localrank.

Here's the Hilltop paper.

And, the Patterns in Unstructured Data (Applied Semantics) piece. This one is VERY long, but of the three, it's probably the most fascinating. It was written before Google bought Applied Semantics and it doesn't really speak in terms of how a search engine would use it, but it does explain the general concepts very well.

Another thing to keep in mind, here, is that a long time has passed since Google got ahold of these things, so their implemenations have likely evolved considerably. But, if you understand the goals and objectives of the search engine companies, then you can pretty easily figure out the direction they have evolved in - and where they will eventually go in the coming months and years.

does it mean those who are already abusing the rel="nofollow" thing to hoard PR, are likely to suffer from this strategy?

Well, sort of. Kinda. Not really - but it's a matter of semantics.

Suffering isn't the right word. You won't get "hurt" by not linking out, nor will you get "hurt" by not having any inbound links. You just won't get any benefits. Honestly, the vast majority of things you can do in SEO is a matter of varying degrees of "plus" and very little that actually provides a "minus". Most of the "minusses" in Google are done by hand when someone catches you being a bad kid.

But, yes, if you are blindly using the rel=nofollow, then you're missing some of the benefit. If nothing else, if you don't link out, you'll likely never become an authority site. Becoming an authority (even if it's an authority on a minute and/or mundane subject) is fairly critical at this point.

And, in the end, as Dr. Jill and crew have been telling us for years - if you are doing anything for the specific purpose of hoarding PR, or for the express purpose of manipulating your ranking in engines (keyword is manipulating), then it's likely going to come back and bite you in the backside sooner or later.

The rel=nofollow has plenty of valid uses. And, if you allow people to post links on your site and you don't have the time or means to police them all, then using it can be a good way to protect yourself. True, you'll lose some benefit from some good links that get posted, but you won't have the risk of a penalty for linking out to a bad neighborhood. It's up to you to decide what's more important.

G.

[OldWelshGuy]

Cheers Grumpus, I have been saying for a while that themed outbound links help the search engines to understand your page better, thereby giving you better rankings, but everyone told me it was because I am a bit nuts

I have found that linking to specific authority pages in sites like .gov etc affect how your site ranks for the chosen phrase (assuming of course that the page you link to is aboslutely on topic).

[Irony]

OWG, I've been saying the same, too. Nobody believes

I'm, honestly speaking, a bit sick of the "hey, you shouldn't have these outbound links on your site, your PR will leak through them" sort of things. But this myth is, perhaps, the hardest one to fight.

[robbinsr]

Grumpus, despite coming in late to this thread, I really appreciate all the work you have done and all of the information you have provided us with. It gives me a lot to think about. I can remember first learning that outbound links steal your page rank away from you, but I just didn't get it at the time, something didn't ring right about that with me, now I am glad to see that it really isn't right. After all, the web was built on interconnected documents, so if we all stopped linking to each other, what would happen to the internet? Would it become a "private net"? That theory also never worked with the authority and hub theories, where if you link to lots of other good quality on topic pages, you will be considered to be an authority for that subject and enjoy a boost to your page rank. Anyways, once again, good show and good on yah! Thanks for all your hard work making a new explanation make sense.

[Grumpus]

Hiya, and thanks.

Careful though - you're mixing terms a bit.

PageRank and page rank are two different things. RageRank is the value that Google gives a page based upon the quantity of inbound links to the page factored by the PageRank of those linking pages. It's that simple, really. It's sort of the base value of a page and, though it's not as important in how a page ranks (this term means "How your page ranks in the SERPs), it's still a factor that is used.

Linking out to other pages doesn't - and never did - steal PR from your pages. There is some grain of truth to the fact that PR will "leak", but it's something that's been drastically overblown. Another single inbound link will replentish the leakage of roughly (I forget the numbers because it's really not that important) a dozen(?) or so outbound links.

Linking out to other sites will NOT improve your PageRank, period. But it will improve how your page ranks.

G.

[Irony]

And can the LocalRank concept be yet another explanation of the aging delay effect?

Why do I think these two are connected:

- they appeared on the scene at about the same time;
- it seems to be very complicated to calculate all LocalRanks properly, so for new sites it will obviously take a lot of time. Until it is done, Google simply has no way of knowing how to rank these sites.

That would also explain why new pages added to older site sometimes experience a similar effect.

Edited by Irony, 28 March 2005 - 04:36 AM.

[robbinsr]

Hey Grumpus, re-reading what I myself wrote, I would have to agree with you, I think I mixed up terms a couple of times! I didn't truly realized there was a difference between PageRank and page rank, so thanks for clarifying that for me, and I believe I may have mixed up hubs and authorities too. If you have a lot of outbound links to good quality sites on one subject, then you become a hub, but if lots of sites link to you on one subject, then you become an authority, right? Regardless, if providing good quality outbound links will help to improve your page rank or how your page ranks, then that is helpful to you on the SERP's, and then if you can get enough good inbound links, which has always been the goal, then that should help improve your PageRank, and combined, both should help to improve your ranking on the SERP's, correct?

[Grumpus]

Robbinsr - I answered that one over here.

Irony - no, not LocalRank. IF LocalRank is calculated preciseley as the paper describes it (things are rarely implemented exactly as they were originally planned, though the core concepts remain true), then LR is something that's caclulated on the fly - specific to each search. You search, a batch of documents come back, you see if any of those documents link to other documents in that batch, and you resort accordingly. It's all done live and there are no static elements involved that would take time to figure out.

But it does take time for TSPR to get calculated. Topic Sensitive PageRank starts with a set of pre-defined topics (most likely, their base for calculating this is the topic categories from the Google (DMOZ) directory). You've got pages in there that fit under a certain topic. Those pages link to other pages. Ideally, those pages will be similarly themed - but you need to check.

NOTE: There are some assumptions below as to HOW they do it, so what you are about to read may not be mechanically sound. The concept of it is pretty close to right though....

Okay, so, you've got your themes and a starting set of pages and each of those pages have links that go out. First, you build a database of words that are common to most or all of the pages that are in that starting category.

First, you crawl the pages and find common terminology. Lots of same words and same phrases but different other words and different phrases means that there are common elements to the pages. Compare those to the words in your database. If a certain percentage of them match up, then that new page is probably talking about the same thing and thus, they are on the same topic. The relevant ones get added to the Topic DB and maybe some new words are added to the word DB. Then you go to the next level. And so on. With each level, the number of pages being compared to each other increases, so it takes longer to do the math. And, by the time your new site initially gets into the Topic Calculation HQ and the math is done to the point where Google even has a general notion of what your site is about, it can take 3-12 (or whatever) months.

There's also the fact that LocalRank uses a lot of resources to do all that math on the fly. So, for competitive and popular search terms, it makes sense to have the calculations all ready. So, rather than using LocalRank for these searches, you tie it in to TSPR. You record a search and look at what's in there. You've got some pages where the TSPR is calculated, and if there's a topic in there, then that search is probably about that topic. So now, you start to figure out how the rest of the pages relate to that topic based upon all the searches that are performed that bring up that/those pages. Blah blah blah, long story not quite so long - it takes a good while to figure all this out and make use of it, too.

Then all these things (and more that I haven't mentioned) are all compared to each other and run through the witch's cauldron, the temperal warp field analyzer, the SingleRussiansLookingForAmericanHusbands ™ Matchmaking software, and maybe take in an episode of CSI on Spike TV. After ALL of that is done - 3-12 months later because, afterall, Spike TV shows CSI a LOT, Google has enough of all those new and spiffy ways of ranking pages figured out for your pages and you can start doing well in Google.

G.

[qwerty]

Topic Sensitive PageRank starts with a set of pre-defined topics (most likely, their base for calculating this is the topic categories from the Google (DMOZ) directory).

Ah, good ol' TSPR. I know the original paper utilized the the top level cats from the ODP, but didn't it also say that they'd need to break those categories down into many subcats in the future for the system to work well?

I don't know to what extent TSPR is being used, and I don't know how much it's advanced since that first paper, but if you look at the set-up page for the Site-Flavored Search, you're starting out with a displayed list of 12 categories, but once you enter a URL and hit the Get Profile button, you're likely to see categories come up in the profile box on the right that are not in the list on the left. So at least for this service, they are clearly taking things further than the TSPR paper did.

[Grumpus]

Yeah. What I was saying was that they probably have gone further than the original paper and might very well be using deeper categories than just the top level ones at DMOZ. It makes sense - you have a DB of pages and topics going down to some pretty specific things. From there, though, yes - you still need to devise even more specific things - and also develop a relational database to how various words used in searches tie back into those topics. Paris is usually about a city in France. Hilton is usually about a hotel. Used together, you're talking about a person - or maybe a specific hotel.

There are so many elements to all of this that I haven't touched on. My goal here, really, is just to give a basic overview of the concepts that people can understand and digest relatively quickly. At some point, I have to omit concepts or glaze over them a bit or people reading these posts will just shut down and not gain anything from them. If they do want to know all the specifics, they can go to the original documents (or, oddly enough, they can have success in their quest for more knowledge by simply typing in some of the fancy terminology we're using into Google).

I don't need to know how my car works in order to operate it - I just need to understand the basic concepts of it and I can get from point A to point B. The same is true here. You really don't need to understand each and every aspect of all this to make it work (though the more you know, the the more refined and effective your efforts will be). Getting the concepts straight is enough to get you rolling and it will help you improve rankings.

G.

[lisphacker]

OWG, I've been saying the same, too. Nobody believes 

I'm, honestly speaking, a bit sick of the "hey, you shouldn't have these outbound links on your site, your PR will leak through them" sort of things. But this myth is, perhaps, the hardest one to fight.

Strictly speaking, that's true if you suppose that the recursive pagerank formula is approximately how PR is computed in the real world. But what's also true is that in doing this, your stock as an information resource rises, and the result for the webmaster, supposing they are linking to valuable resources, is a net gain in the total "Google stock" of your website.

Like Grumpus reiterates, it's all a matter of net gain. Some things might "hurt" you a little on one metric, but at the same time, help moreso in another, hence net gain. I always tell people to "think like a librarian" when building their site, even if it's not primarily an information site.

[Irony]

Strictly speaking, that's true if you suppose that the recursive pagerank formula is approximately how PR is computed in the real world. But what's also true is that in doing this, your stock as an information resource rises, and the result for the webmaster, supposing they are linking to valuable resources, is a net gain in the total "Google stock" of your website.

Like Grumpus reiterates, it's all a matter of net gain. Some things might "hurt" you a little on one metric, but at the same time, help moreso in another, hence net gain. I always tell people to "think like a librarian" when building their site, even if it's not primarily an information site.

Exactly PR leakage may exist (to my calculatons, though, it's way too tiny to bother about). But it would be very stupid if engines started leaking actual SERP rankings for linking to relevant sites. I'm sure they understand it. The problem is, many people don't. They don't listen to any arguments you may have, either.

Grumpus, thanks a lot for clarifications.

[Hyperformance]

Excellent thread to everyone! Thanks Grumpus for all the clarifications..

I always knew your outbound links had importance, even if for the simple fact that my site got related to (associated with) the right Industry Hubs. (Those major players in any category that you NEED to be associated with and become a part of...)

Great stuff here!