33 replies to this topic

[BBCoach]

[Split this interesting side conversation from here. - Jill]

Since the search engines don't really "theme" sites, there's really nothing you need to do. They will probably still think it's a humor site because of the links pointing to it, however.

Jill I'm not quite sure what you mean by that because SEs do group your site in with sites like yours. What about [related:www.mysite.com]? If that's not "themeing," then I'm clueless. To me the evidence that they do it is Adwords. My hobby site was grouped in a "human resources" area for a long time due to what I was doing with it for period of time, but after my refocus and changing of it's theme it's spot on now. I think they do it and it's a combination of algos and human reviewers. Also, if memory serves me, G has a patent filed for doing just that. If you think I'm wrong or looking at the wrong things, then what purpose could the query [related] have?

Why not just buy a new domain? They're only $10 a year or less.

I agree with just get a new domain name.

[Jill]

BBCoach, can't say as I've analyzed the "related" sites link in many years, so I don't have an answer for you.

At first glance, it appears that they're showing pages that have the same basic words as the page you're putting in the related field. To me that isn't really theming, but again, I'd have to look at it more closely. Will try and look into those results more closely and get back to you!

[Randy]

If you look deeply and long enough at linking relationships --the way Google does-- the related sites will make a lot more sense.

In many cases it's pretty easy to spot a trend because the sites in questioni may have similar words in the content and also have links from some of the better known directories out there that happen to have them in the same category. Other times it's harder to spot the linking relationship because there may be one site that links to 5 of the related sites and another site that links to 10 sites from the same page/section, of which 3 are the original 5.

In other words, it's more than just about the words on the page. You'd also need to factor a broader range of linking relationships between multiple sites into the equation.

[BBCoach]

In other words, it's more than just about the words on the page. You'd also need to factor a broader range of linking relationships between multiple sites into the equation.

Randy, that's what I've been noticing for the last few years without linking relationships with several of my clients. I.E. my primary client is in the hunting/shooting industry. If you do a [related:] query most of their close competitors are listed. Same with all of my other sites. My hobby (testing) site exhibits the same traits. It takes awhile when you re-theme a site, but from what I've been noticing the SEs (Google in particular) will group your site into the same theme/category. I think if using a [related] query brings up your direct competitors and similar blogs and/or forum posts, then all of them have been themed or categorized whether it's due to a directory listing, algos, linkings or human reviewers. Why? Because we definitely don't link back to one another. Of course I don't have any hard evidence, but the circumstantial evidence is almost deafening. It appears that similar web sites are grouped together (themed) in the SEs. At least that's my observation. I'm not saying anyone is wrong or right or that there is or is not a benefit, only that this is something that I've been observing over the last 3-4 years in particular.

Logical question. If all of the SEs stated goal is to "organize the world's information," then would it not make sense that SEs put like sites (content providers) in the same categories when searching for related information? After all they are doing that with web sites in PPC campaigns. I couldn't imagine having to filter out hundreds of off-topic ads from my web sites because G/Y can't figure out the theme/category my sites are in. No hard evidence, but I believe from my observations that theming/categorizing of web sites is going on in the SEs at some level if not at a higher level.

[Randy]

I agree with the basics you've noticed BBCoach, however there are always those wonky cases that throw a monkey wrench into the equation of what we think we see. And of course there is always the argument/differences in what word of "theming" really means.

Let's take my little personal domain as an example. And I'll grant you right from the start that there is no well defined theme that runs throughout, me being a bit scatter-brained at times and having a wide variety of interests.

According to Google my site is related to Google.com, MSN.com (I'll give them that one even though the real search url is now Live.com and not MSN.com), Yahoo.com, Ask.com and other sundry places in the Search space like Search Engine Watch, the Yahoo Search Blog, Search Engine Guide, and of course the HR Forums.

Do I link to some of those places? Yes. Many of them in fact. Are there links in some of those places that point back to my site? Yes, some do. Could one call the general theme of my personal site be considered one tailored towards things about the Search market? Yes, for some pages, No for others. I blog about a lot more than just about Search, since I do a lot more than just search.

Google does in fact try to incorporate some of the other things into the Related search. They also list a couple of places like f-secure.com and and safer-networking.org because I sometimes write about server and overall network security. I think we can all agree those two themes have little to do with each other, but at least they try to show a better rounded picture.

I can tell you though that they're missing things/sites that I would expect to be included. For instance missing from the list is WebProNews where David Leonhardt has an SEO Tools article he penned last year that mentions and links to a little app I wrote as one of his Top 10. It's far from the only example, but it's an example. If they truly consider my personal site to be a Search theme, why wouldn't they show other obviously Search related sites that link to my site as being related? If a human were reviewing things it would be a natural connection, I think.

Now on thing that I'm going to be interested to watch is what information Google actually uses to come up with Related sites. Or more specifically if they'll use GoAn data to give a clearer picture. They're not yet, but I just installed GoAn on this site a month or so back, so it's too early to hazard a guess if they're using its data or not.

The reason I raise the question is that if I look at my GoAn stats I can see that almost half of my traffic coming from Google is people searching for something having to do with ImageMagick. The why of that is simple. I wrote a blog post awhile back laying out how to install ImageMagick with support for jpeg, tiff, png and gif images because there was no decent documentation out there for server admins on how to do it. Apparently there's little competition for this type of very targeted phrase because my little site seems to rank well for lots of terms surrounding the space. Many times ranking better than the ImageMagick site itself! But people are searching for it because I'm getting traffic.

So will a single blog post make the site Related to imagemagick because such a hefty percentage of overall visits according to GoAn end up at my site for imagemagick installation searches? If Google were using the GoAn data at all for things like Related searches I would expect to see some indication, considering ImageMagick searches bring me almost half of the (measely) traffic that hits my personal domain. I'm not seeing this though.

If I were Google I'd be using this data, after all it's directly on target and incorporates more than just Google searches so should give a better overall picture. In fact, since I have more than one site in my GoAn profile I'd be listing those other sites as somehow related also. While the subject matter may not be related, the sites do have a considerable connection with each other. Namely me.

At the end of the day I'm left with that a site's Theme is something that's very difficult to narrow down to a single thing, or even to several things, without having a lot of information on hand. I'm also left with the fact that it's a lot harder for a computer to make such assessments than it is for a person because we're wired differently. They may get it some day, but try as they might that day isn't here yet.

[BBCoach]

I agree with the basics you've noticed BBCoach, however there are always those wonky cases that throw a monkey wrench into the equation of what we think we see. And of course there is always the argument/differences in what word of "theming" really means.

I agree with the "wonky cases," but I believe it's due to the algos attempting to do it with not enough human intervention. Also, there doesn't need to be an argument about what theming really means because the discussion really should be about categorization.
Definition of theme is:

1.a subject of discourse, discussion, meditation, or composition; topic: The need for world peace was the theme of the meeting.

2.a unifying or dominant idea, motif, etc., as in a work of art.

3.a short, informal essay, esp. a school composition.


Web sites can be "themed," but for indexing/cataloging purposes, what category are they in? What industry and/or interest, specialty, sub-specialty etc. does your web site fall into more than anything else or equally? Even though your web site branches out in many directions, there are some major categories that it definitely falls into. Computers/Advice, Computers/Programming, Computers/Graphics, Computers/Internet Marketing, Computers/Programming/Web Site Development, Computers/Network Security and Red Lederhosen. Sorry. Couldn't resist.

I'll be the first to say that the SEs have a LONG way to go to get their arms around most of the web sites to properly categorize them, but they're getting better at it from my observations. So is this important? I dunno. I'm still testing my observations to determine whether it has legs or not. The question that I'm seeking an answer to is, "Do SEs really care to group like web sites together or are they content with grouping the web sites' individual pages, and is there a benefit to them by grouping web sites?" If they do, then how can I best take advantage of it? Right now I can't tell that there's more than a 50% probability that it matters. Not enough to conclude anything. Logically and programmatically I think it makes sense to group web sites into major categories and then use any sub groupings on the web site to relevantly categorize individual content pages (as the SEs confirm by having specialized searching for books, government info, scholar info, maps, etc). It's only one of the things I've been researching for the last couple of years, and it will probably take a few more before I either throw my hands up in surrender or discover something useful from this exercise.

Now on thing that I'm going to be interested to watch is what information Google actually uses to come up with Related sites. Or more specifically if they'll use GoAn data to give a clearer picture. They're not yet, but I just installed GoAn on this site a month or so back, so it's too early to hazard a guess if they're using its data or not.

I think it's one of G's tools to help facilitate this. I can't think of a more automated (inexpensive) way of gathering or correlating that information, but it's flawed because not everyone uses GoAn.

[Randy]

Sorry. Couldn't resist.

I'd have taken the shot myself!

I agree with the "wonky cases," but I believe it's due to the algos attempting to do it with not enough human intervention.

I wonder if it's a human intervention thing, which would probably be cost prohibiitive with the number of sites out there if they were doing it with internal staff, or simply not enough information. Or perhaps they just don't have the algos nearly perfected yet. Probably a combination of all of the above and more.

I think it's one of G's tools to help facilitate this. I can't think of a more automated (inexpensive) way of gathering or correlating that information, but it's flawed because not everyone uses GoAn.

Agreed. But for those sites that do in fact use GoAn and give up all of that data for free, why wouldn't they use it? It quickly becomes a question of punishing some because not all provide enough information, or use what you have and leave those who don't play along out of the loop. It's an interesting question, one I wonder if they've answered internally at The Plex yet or not.

And just to take us off topic again...

All of which sort of leads back into the recent discussion we had here at HRF of whether or not Google will use things like bounce rates and conversion metrics as a factor in their per keyword phrase or per theme ranking algo for those sites that do in fact use GoAn and/or their Website Optimizer. That's really why I'm looking for a clue to see if they're using any of the data in their ranking formula or not.

[BBCoach]

I wonder if it's a human intervention thing, which would probably be cost prohibiitive with the number of sites out there if they were doing it with internal staff, or simply not enough information. Or perhaps they just don't have the algos nearly perfected yet. Probably a combination of all of the above and more.

Agreed.

All of which sort of leads back into the recent discussion we had here at HRF of whether or not Google will use things like bounce rates and conversion metrics as a factor in their per keyword phrase or per theme ranking algo for those sites that do in fact use GoAn and/or their Website Optimizer.

I can't come up with a business reason to not use it, but would ask a question on how to most effectively use it and to keep it in the damp dark halls of secrecy. That is if I were in the Plex. Properly sliced and diced it would appear that it would have tremendous benefits to accuracy and relevancy for a search.

[mcanerin]

Rand and I had this argument (though quite civilly) on a flight from Shanghai to Xiamen once. I'm in the "no-themes" camp and he's not (or wasn't at the time - I don't want to put words into his mouth).

He argued that a) it just makes sense, b) he's seen results that support it, and c) that Google had in the past acquired a company that had software that automatically created taxonomies for sites.

I argued that a) it doesn't make sense (what "topic" is Wikipedia about?) b) I've seen results that don't support it, and c) Google at one point wanted to automate the Google Directory and that software was probably intended for that.

I'm afraid it was a stalemate - neither of us changed our minds at the time, but I've been thinking about it recently and now have a different opinion which I think may explain what both of us have seen.

I think Google may assign a "category" to certain sites, like pharmacy, gambling, etc, but only when there is a secondary process that requires it. For example, it would not surprise me to find that adult sites may be assigned to an "Adult" category in order for Googles "Safe Search" function to work. Doing this ahead of time would be efficient and fast.

I suspect that there may be several categories effective on a site at once, as well, so a site could technically be assigned a country category and be a gambling site, for example - both of which would have special processes in place. Think of it as "tags" like in social media - you don't add tags for every single thing about a site , but rather the key things that may be of interest. This would allow you to sort very quickly.

However, I also believe that assigning all sites a traditional taxonomy or themes would be slow, fairly useless and a waste of space, not to mention nearly impossible. If you have ever worked as an editor in a directory, you'd find very quickly that there are a lot of sites that can legitimately fall under more than one category or theme - often dozens.

There is also the case that you need to ask whether it makes more sense to categorize on a page level, or site level. Many sites can have a lot of different "themes" in them. That's a lot of processing. For what? Is it really necessary to know that one site is in the category of Gardening:Posies and that another is in Gardening: Pansies? Wouldn't a simple keyword check accomplish that?

No, I think (and it is what *I* think, I'm not stating it as fact) that search engines will assign tags or categories that help them solve problems and speed things up, but not assign "themes" per se.

Ian

[Jill]

Good stuff, guys! Get ready for some additional thoughts as the newsletter is about to hit the streets with this thread as the highlighted one for the week!

[piskie]

I think Google does actually Theme sites. This may split a site as covering a number of related themes.

I got this idea in my head from a conversation a few years back when talking to a Gman about hosting sites on the same Server and same IP and then linking them together even mutual 2 way links.

He said providing they were "Like Minded Sites" there would be no problem.
He did say however that if it were plain old fshioned link spam, by being co-located any spamy relationship would be picked up pretty quickly.

Now to my mind for "Like Minded Sites" to be used as a yardstick would need some degree of Theming.

In the same conversation, we then got into the subject of Word Stemming with an explanation of why theming would be important to facilitate word stemming and semantics.

[projectphp]

The first rule of logic, and the best known argument against God, is that until you define a term, you can't argue about its existence.

Theming has no definition, and as such, arguing whether Google use it is a step we just can't take.

IMHO, theming is either defined so broadly it is a truism (Google decide what sites are about and rank them accordingly.. um, yep, that about defines what a search engine does!) or it is used as some mythical thing that has some implementation we aren't sure of, but we are sure exists. Even if they do use something similar, given I haven't seen the term used outside SEO sites, it likely it called something else.

That Google tags sites like Ian suggests is, IMHO, pretty much certain, because we have blog search, Australian sites etc etc. SE are likely to label sites in all manner of ways. Whether they do this using a taxonomy like system, e.g, categories, is something that I don't think anyone can show with much certainty.

IMHO, it is far more likely that "themes" as people understand them are different to the way an algo would work. Themes are more likely to be decided using shared links than any taxonomical decision about where a site fits. To go to teh effort of defining all the cats there are, and then pidgeon holing sites seems a lot of work to me, especially when that also needs to be linked back to keywords, e.g. a search for nike is a shoe search.

Seems to me there are likely faster and better ways to categorise sites than a strict taxonomy.

[mcanerin]

One other thing: The "Related Sites" search is not based on some sort of theme.

It's based on shared backlinks.

The assumption is that if two sites have similar backlinks, then they are probably in a related area. When I first launched my SEO site many years ago, I did the same link building campaign on my own site that I did on my first few clients.

The Result?

When I did a related sites search, I found that I was in the proud but lonely group of "whitewater rafting, coffee roasting, real estate selling search marketers". Later, of course, as I collected a lot more relevant backlinks, the related sites result became more accurate and focused. I still use the function to do competitive analysis for client competitors - it's a great way to find people who own link farms or use SEO's that have link farms.

Randy, that's why you get the results you are seeing. The kind of people who link to you are also the kind to link to Google, for example. Generally, it works fairly well.

But it's not related to "themes", and is yet another reason I doubt the concept. Why use backlinks when you have themes to find related sites?

Ian

[projectphp]

The biggest nail for theming, or any sort of taxonomy, is that it isn;t comparative. What the "best" site in a DMOZ cat is not clear, for example.

Using any sort of theming won't help ranking, it would only help qualification, and there are already a ton of useful tools for that.

[glengara]

I've found "related" has changed quite a bit, in the past results were based on links so where linkage was "promiscuous" the results had no discernable relation to the sites' "theme", today they seem a lot more focused.

It goes back to the early days and "Google Scout", when the guys still believed links would invariably be on-topic and so "related" :-)