23 replies to this topic

[greggb]

Hi all. I don’t visit this forum very often, and I don’t keep up on SEO happenings. So this might be old news to many of you, but I thought I’d bring it up anyway.

I’ve noticed recently, monitoring SE rankings for a few websites, that MSN’s algorithm really stands apart from the rest of the major SEs’. In particular, it seems to have a real affinity for pages—lots of pages. By that I mean pages with different filenames. I would venture to guess that MSN places some emphasis on keywords within filenames (just a guess). I know for sure that MSN considers the number of pages with different file names to be an indication of quality and/or relevancy. Meaning, it likes a website with, let’s say 65 differently named pages, more than a website with 5 differently named pages. Relevancy of those pages is probably a factor. At any rate, an e-commerce website with 65 product pages (i.e., product_43.html) will rank higher at MSN than a website with one dynamic page for displaying products (i.e., item_display.php?itemnum=43). There’s no doubt in my mind about that.

BTW, this isn’t the result of a “We won’t index your dynamic pages until you have a certain status” thing. I’m talking about websites that get a full spidering and indexing of anything from Cold Fusion pages to CGI scripts.

Also, I wonder if it’s any coincidence that most of the major search engines (save MSN) normally have very similar rankings for the same website. At least, that’s something I’ve noticed with the websites I’ve been monitoring. For the last two weeks I’ve seen a string of 4 #7s or so, a #8, a #6, and then of course a #1 for MSN. Do you suppose that some of the SEs take little peaks at Google SERPs every now and then? Kind of like, “Our algorithm says to rank this website at #12, but Google’s ranking it at #4, so let’s rank it at #5 so it doesn’t look like we’re copying.”

One more thing. I’ve noticed that pages that are updated frequently get frequent indexings by Google, where pages with less frequent updatings get less frequent indexings. I’ve been getting indexings on one of my websites at a frequency of almost every other day. That is, one of my frequently updated websites gets an indexing every other day, and sometimes two days in a row, most of the time. I’m a month out from getting an indexing on a website I manage that hasn’t been updated for a while.

Do you suppose that you get google brownie points for having a frequently updated website, where there’s a frequent and significant change in content (but the content still stays relevant)?

Anyway, I just thought I’d mention a few of my observations. Call it food for thought, or something like that.

Gregg

[Haystack]

A quick thought on one of the theories: If I was writing a search engine, I don't think I'd give extra brownie points based on frequent page changes. I don't see how that correlates with relevancy.

[greggb]

A quick thought on one of the theories: If I was writing a search engine, I don't think I'd give extra brownie points based on frequent page changes. I don't see how that correlates with relevancy.

True, but SEs obviously place some emphasis on quality (i.e., link popularity). A frequently updated website might be considered an indication of quality. And I'm not just talking about the "last modified time" attribute of the file itself. I mean a genuine change in content of the pages on the website. That might score a point or two for quality.

Gregg

[Haystack]

Each change would bring a change in rankings, but I don't see why the changes would help you rank more competitively for a given search phrase. For example, if your site was trying to rank competitively for the term, "Blue Widgets" how would changing the content to something else help you rank competitively for that term? Or, are you suggesting that you should change it, then change it back, and at that point you'll rank higher than you would have in the first place due to frequent changes? If so, I haven't seen any evidence to suggest that search engines look at such a metric.

[vmills]

Hmm, I was coming to somewhat different conclusions about MSN and stopped by the forum to check my assumptions. I am reviewing a real estate site that is nowhere to be found in Google and Yahoo, but that ranks pretty well in MSN. The site has a PR3 (not many quality links), but the keywords aren't very competitive, and it seemed like it should show up somewhere in Google or Yahoo. The biggest limiting factor I can find is the site was built in such a way that only the home page has been indexed (interior pages are framed). My guess is that Google and Yahoo filter out 1-page sites (or penalize them - whatever term you want to use), but that MSN weighs on-page factors heavily and doesn't give much weight to the number of pages. Whaddya think?

[greggb]

Each change would bring a change in rankings, but I don't see why the changes would help you rank more competitively for a given search phrase. For example, if your site was trying to rank competitively for the term, "Blue Widgets" how would changing the content to something else help you rank competitively for that term? Or, are you suggesting that you should change it, then change it back, and at that point you'll rank higher than you would have in the first place due to frequent changes? If so, I haven't seen any evidence to suggest that search engines look at such a metric.

Take an auction website, for example. The auctions on the home page change by the hour, and even more frequently than that. The content on the home page is very different from day to day, because of the way auction listings are rotated. Of course, most of the auctions are relevant to the theme of the website in some way or another. So what you get is constantly updating, relavant content.

I'm only wondering if constantly updating, relavant content might score points with the SEs. It seems to me that when you get into statistical modelling, you'd find that websites that are frequently updated tend to be higher quality than websites with pages that sit for days or months.

Gregg

[OldWelshGuy]

Vmills it is is probably mainly down to the fact that Yahoo & Google place more value on links, boith internally and externally. framed pages end up disjointed, orphaned etc. MSN is stil a bit naive when it comes to spam IMO.

[DanThies]

A quick thought on one of the theories: If I was writing a search engine, I don't think I'd give extra brownie points based on frequent page changes. I don't see how that correlates with relevancy.

Ed,

Assuming that you were delivering all 10 search results with the same algorithm, I agree that rewarding "freshness" would not necessarily be the best approach. I personally don't believe that any of the major search engines is using just one algorithm or weighting method for all of the results on the first page.

If you wanted to give your searchers a variety of results (not "all recently changed" or "all old and trusted") you might very well want to use freshness or frequent updates as a factor in your algorithm. You might want some hubs, some authorities, some shopping sites, some fresh, some old, etc. depending on the search term.

[greggb]

Ed,

Assuming that you were delivering all 10 search results with the same algorithm, I agree that rewarding "freshness" would not necessarily be the best approach. I personally don't believe that any of the major search engines is using just one algorithm or weighting method for all of the results on the first page.

If you wanted to give your searchers a variety of results (not "all recently changed" or "all old and trusted") you might very well want to use freshness or frequent updates as a factor in your algorithm. You might want some hubs, some authorities, some shopping sites, some fresh, some old, etc. depending on the search term.

Hmmm... I wonder how you'd you go about ranking results if you weren't using the same SE algo for all the results? I think SEs use the same algo... I think they'd pretty much have to, or they'd get into serious logistical problems. But I'd buy that the search phrase itself has some bearing on the results turned up. SEs have to know that certain queries made are people looking for information, while others are people looking for products, etc.

I think when you take a hard look at statistics, you'll find that websites that are updated frequently tend to be higher quality than those that aren't. I'm not saying it would make or break a website, but I'd be pretty shocked if SE algos didn't give a little bit of weight to something like this.

[DanThies]

Hmmm... I wonder how you'd you go about ranking results if you weren't using the same SE algo for all the results?  I think SEs use the same algo... I think they'd pretty much have to, or they'd get into serious logistical problems.  But I'd buy that the search phrase itself has some bearing on the results turned up.  SEs have to know that certain queries made are people looking for information, while others are people looking for products, etc.

When you understand how they come up with search results, it's relatively easy to conceive of a base algorithm delivering the top n results (i.e 1000 results as Google does), then applying additional factors to reorder those results to produce a more varied SERP with different types of information resources.

As you already stated, SE's can pre-process the search query before fetching results. For example, if a 3-word search query is entered, the SE can quickly calculate an E/F ratio* to determine whether the search term is commonly used as a phrase, or if it's just 3 random words.

If the weight placed on "keyword proximity**" were proportional to the E/F ratio, then users would get the benefit of "implied quotes" around phrase searches without actually having to type quotes, and better search results when they're just adding words to a query in an effort to narrow their search.

The latter is just one example of how a search engine could modify the weighting/algorithm in a query-dependent fashion, but that's "old hat." A lot of smart people work on improving the user experience at search engines. Assuming that search engines must take a one size fits all approach to every SERP is unrealistic.

* E/F Ratio = comparing the # of results from an exact phrase search (with quotes) to the find all search (without quotes). For example, on Google "search engine optimization" in quotes returns 7,220,000 results, without quotes returns 8,270,000 results. 87% of the time these words are found on a web page, it is used as a phrase.
** Keyword Proximity as a factor in retrieving search results: how important is it that the words appear close together? If the user is searching for a phrase, it's very important. If they're using 3 random words, it's less important.

[sweepthelegnate]

A lot of smart people work on improving the user experience at search engines. Assuming that search engines must take a one size fits all approach to every SERP is unrealistic.

that's why simply building the best site you can for your visitors (ie lots of great content and good assesibility) will get you a long way with the SEs or will eventually


nathan

[greggb]

When you understand how they come up with search results, it's relatively easy to conceive of a base algorithm delivering the top n results (i.e 1000 results as Google does), then applying additional factors to reorder those results to produce a more varied SERP with different types of information resources.

I understand how SEs come up with results, and that's why I said it would be a logistical nightmare to use different algorithms in the same results set. Kind of like, "Algo 1, you go find some results and report back to me. Algo 2, you go find some results. When you guys get back we'll put our heads together and figure out which results we're going to put where". The problem in using different algos is that the left hand needs to know what the right hand is doing. Maybe I misunderstood what you meant when you said different algorithims?

As you already stated, SE's can pre-process the search query before fetching results. For example, if a 3-word search query is entered, the SE can quickly calculate an E/F ratio* to determine whether the search term is commonly used as a phrase, or if it's just 3 random words.

Actually, I was referring more to statistics. I'm not saying that it's totally impossible for inanimate object to use a simple two-variable formula to determine if three words from a natural language are intelligible or just random (though that sounds a little bit sci-fi). I just think a more practical solution might be to check the old query DB, and compare a particular phrase with what people tend to be looking for when they query that particular phrase. Like when someone types "information on SEO", statistics say that they're looking for information on SEO. And when someone types in "information technology companies", statistics say they’re looking for companies who provide IT services, not information on "technology companies". IMO these kinds of distinctions can only be made by examining statistics.

If the weight placed on "keyword proximity**" were proportional to the E/F ratio, then users would get the benefit of "implied quotes" around phrase searches without actually having to type quotes, and better search results when they're just adding words to a query in an effort to narrow their search.

The latter is just one example of how a search engine could modify the weighting/algorithm in a query-dependent fashion, but that's "old hat." A lot of smart people work on improving the user experience at search engines. Assuming that search engines must take a one size fits all approach to every SERP is unrealistic.

I agree that the algo you're describing is probably very old hat. I also agree that SEs are not using a one size fits all approach. But I tend to think that SE algorithims are more linear... that they have more simplicity than multiplicity, like any good algorithm does. That's why I tend to think that an intelligent SE would examine a query, venture a guess as to what the querier is looking for, and do its best to provide the querier with what he's looking for. Rather than being indecisive and throwing a bunch of different stuff out there, hoping it might get it right by chance.

This has been an interesting discussion… thanks for the reply.

Gregg

[Clintorius]

I think when you take a hard look at statistics, you'll find that websites that are updated frequently tend to be higher quality than those that aren't.  I'm not saying it would make or break a website, but I'd be pretty shocked if SE algos didn't give a little bit of weight to something like this.

If such a superupdated site is of high quality in it self and therefore ranks well - why would the SE put additional weight on it?

[greggb]

If such a superupdated site is of high quality in it self and therefore ranks well - why would the SE put additional weight on it?

Since when does having a high quality website mean that it will rank well? There's no "therefore" after high quality.

[DanThies]

Frequently updated also means that someone is working on it. This describes a high percentage of websites, period. A site that someone is working on is more likely to be optimizing content, building links, etc. than a site that hasn't been touched in months, no? Assuming cause and effect just doesn't make sense.