58 replies to this topic

[Michael Martinez]

Right... got that... I do this stuff for a living... so if the search engine places emphasis on the first word in the query, wouldn't you say that the word order is important? As opposed to "not really important?"

We're really talking about two different concepts.

The terms in a Find All mode query are treated as independent entities, but the query is treated as though it's a variation on the first term.

Hence, proximity and word order are not really important with respect to the query expression. Relevance to a Find All mode query is not predicated upon word order or proximity.

An exact find query looks for a chain of words strung together (not really an expression in the intuitive sense). Proximity and word order are therefore important.

I can illustrate how Find All differs from Exact Find with this example: A variation on the Find All mode, makes use of the plus (+) operator to require that a specific keyword be included in the listed documents (inbound link anchor text is allowable). So, you can query for:

term1 +term2 term3 +term4

You often get results that don't include term1 or term3, or they may be in any order where term2 and/or term4 occur first.

The query tool will prefer the term that is least common to the common results set, since the rarest word in your query expression is more likely to be significant than the most common term. You can test this principle by running queries for expressions with 1 long word (3 or more syllables) mingled with 2 or 3 1-2 syllable words. The longer word (like "insubstantial") will be favored, although other factors (such as frequent occurrence of the shorter terms) may outweigh the natural rarity of the word.

You'll see different SERPs (usually) but the word order and proximity will vary within the SERPs. Look at the variations returned for "insubstantial hair loss", "subsequently placed aside", and you'll see what I'm referring to. Naturally, if 2 or 3 words are normally used to form an expression together, there is a higher probability that their co-occurrence will impact the SERPs (as with "sea monkey")

So Find All does not mean "find all documents where these terms occur in this order" much less "find all documents where these terms occur next to each other".

[DanThies]

Yes, Michael, I know what "find all" means. I also know that you're making some big assumptions about how actual (vs. theoretical) search engines operate on user queries, that you are making contradictory statements, etc.

But as you have done for so many years, once again you have worn me out. The field is yours.

[Michael Martinez]

Yes, Michael, I know what "find all" means. I also know that you're making some big assumptions about how actual (vs. theoretical) search engines operate on user queries, that you are making contradictory statements, etc.

No, Dan, I'm not making assumptions. One need only look at the results for random searches (such as those I provided) to see that there is no direct correlation between a Find All query and word order/poximity in the results of the search. Any word can come first in the document, or last, regardless of which order is chosen in the query.

Hence, I stand by what I wrote originally, "Most people search in what is called FINDALL mode, where word order and proximity are not really important".

The technical reasons don't have to be explained in excrutiating detail for that point to stand.

You could just as easily have pointed to Benefits of Google's search to strengthen your case for proximity, but there are plenty of searches that reveal proximity is not nearly as important as other factors.

3. The position of your search terms is treated with respect.
Google analyzes the proximity of your search terms within the page. Google prioritizes results according to how closely your individual search terms appear and favors results that have your search terms near each other. Because of this, the result is much more likely to be relevant to your query.

Jeremy Zawodny pointed out discrepancies between what Google says and does in 2003. I'm just pointing out one more, but the results can be seen elsewhere, too.

[DanThies]

OK, Michael, what the heck... quotes from you:

The terms in a Find All mode query are treated as independent entities, but the query is treated as though it's a variation on the first term.

FINDALL mode, where word order and proximity are not really important

So which is it? Word order doesn't matter, or the first word is more important?

there are plenty of searches that reveal proximity is not nearly as important as other factors

Yes, it will be different depending on the query. Word order won't matter if you just stick random words together (insubstantial monkey test). However:

Naturally, if 2 or 3 words are normally used to form an expression together, there is a higher probability that their co-occurrence will impact the SERPs

One of these days, I'll run another chart to map E/F ratios vs. changes in search results. Last time we did that, we had a pretty clear indication that FINDALL doesn't necessarily work like textbook FINDALL - it's sort of like "implied quotes" around search queries that are naturally used as phrases.

It is of course difficult to sort out the influence of on-page text vs. inbound links and anchor text, but natural links are yet another reason to consider natural word order in headlines, titles, copy, etc.

<added>Re: Jeremy's post, he'd have seen similar amazing rankings by replacing "for" with another stop word, like "Schwarzenegger the governor" - is Google observing that the query has a word between Schwarzenegger and governor? Maybe "the position of your search terms is treated with respect" in some fashion?</added>

Edited by DanThies, 04 August 2006 - 01:01 PM.

[Michael Martinez]

OK, Michael, what the heck... quotes from you:
So which is it? Word order doesn't matter, or the first word is more important?

First word in the query versus word order in the document. That's not an either/or concept.

there are plenty of searches that reveal proximity is not nearly as important as other factors

Yes, it will be different depending on the query. Word order won't matter if you just stick random words together (insubstantial monkey test).

And it won't matter if you don't stick random words together. If Find All searching were dependent on word order, there would be no need for exact find. Examine the results (which you and I know will vary over time) for keyword marketing research (which I chose because I see active searching on this expression).

The phrase "keyword market research" does not occur in the text (according to a manual search I just performed) of the second page listed, although all 3 words are listed and "market research" is used several times.

It does occur in the first, third, and fourth entries. It does not occur in the fifth entry. Some of the pages have "keyword research" and some of them have "keyword marketing" (allowing for stemming, although "market" appears on a couple). Some of the pages put "market/marketing" first, some put "keyword" first.

The top ten results therefore are not ordered on the basis of keyword proximity and order.

However, no one has to take my word for it. Here is the quickest authoritative source I can provide for this. I don't feel like searching the technical literature for further citations:

Unlike OR (ANY) -in which retrieved documents must contain any of the query terms- or unlike EXACT -in which the documents must contain a queried sequence of terms- in FINDALL the documents retrieved must contain all query terms, regardles for order and proximity (1, 2).

Dr. E Garcia (aka Orion)
http://www.miislita....g-patterns.html

While he may be touting a textbook position you distrust, I'm inclined to accept his proposition on the basis of his professional credentials nonetheless. But there are many queries that demonstrate word order and proximity are not very important. Skipping ahead...

...It is of course difficult to sort out the influence of on-page text vs. inbound links and anchor text, but natural links are yet another reason to consider natural word order in headlines, titles, copy, etc....

I agree with you.

Further, idiomatic usage will have a heavy impact on the probabilities of what is returned for search results. Natural language expressions still seem to occur more frequently than randomly patterned words. However, there are plenty of SERPs which show subsets of query expressions separated from other portions of the query expressions (the queries I provided as examples demonstrate that).

<added>Re: Jeremy's post, he'd have seen similar amazing rankings by replacing "for" with another stop word, like "Schwarzenegger the governor" - is Google observing that the query has a word between Schwarzenegger and governor? Maybe "the position of your search terms is treated with respect" in some fashion?</added>

Maybe. Google hasn't been indexing expressions for all of its career. They don't disclose much about how they process expressions.

But it's still clear that Find All is not an expression-based query, given that their query guidelines indicate expression searching is to be initiated by including quotes (") around the expression.

[DanThies]

Mike,

Just to be clear, you believe that if:
1) I write two web pages, one using keyword research marketing and another using keyword marketing research in title and text
2) These two pages would be treated, by all search engines, as equally relevant documents for either search query

Is that accurate?

Dan

[themezoom]

Hello,

Gee. I would say. NO. (To the last question that is).

Although I hope we can ask some questions without getting hit by a flying chair.

We wanted to thank you for the very thorough geekspeak and reduction of the topic of FIND ALL and EXACT MATCH.

This thread is being followed closely by our newbie forums, because of the topic of "awkward phrases" which was the original cry for help from a newbie in this particular forum. So the thread was announced etc. So there are some guys watching the SEO deathmatch.

Because this is also a topic for my group, Sue and I have read your "debate" with great interest- getting past what seems like 'bad blood' in order to glean the truth for a few hundred folks.

The original keywords Adrian asked you about were:

bowel cancer symptom
early symptom bowel cancer
sign and symptom of bowel cancer
cancer bowel sign symptom
advanced cancer bowel symptom

The real issue is that both of you seem to be "right" about many of the same things- and it seems that the original purpose (to help the novice mindset, if not the specific newbie) should come back around.

1. What would you tell a newbie about the issue of "awkward phrases"?

Our position has been to encourage them to add natural, context-friendly expressions where it makes sense and the proximity and rarity will (generally) take care of itself. This is GENERALLY what we understood Michael to be saying.

If after building traffic and strong themes they get multiple "artificial" queries showing up consistently in their web-log files, they may want to consider using them in some sort of on-page context, but even better they may want to drill into the awkward terms and find more coherent phrases and themes with higher trafficked expressions.

Comments?

What is more important than getting lost in the forest (website page) with the trees (keyword order, rarity and density) is probably comprehending the overall theme of the website and how that natural or awkward phrase fits into the larger whole.

Now I admit that Michael has made me seriously reconsider some of my own notions in regards to rarity. I will need to digest this for awhile, because now we are looking at rarity as it relates to overall theme. (Kewl).

I could be wrong- and although I am dazzled by you guys technical reduction of the "query order" to "page order" contradiction-

. . . it still seems like the real issue that Dan brought up was diluted and then brought back:

QUOTE(DanThies @ Aug 4 2006, 11:54 AM)
OK, Michael, what the heck... quotes from you:
So which is it? Word order doesn't matter, or the first word is more important?


First word in the query versus word order in the document. That's not an either/or concept.

It seems to me that this encapsulates the essence of the "argument". And Michaels answer should be brought back "down" to the real world issue:

What do most search users type in? When does it make PRACTICAL sense to use awkward phrases throughout your website. I have about as much interest in "forced" awkward phrases as I do in the Overture suggestion tool.

NO REALLY, "Canadian Company Engine Optimization Search" is a real keyword. (As if).

The results from this query comes up differently between exact and find all obviously. There are a ton of sites that have it on the page as an "exact match" because they pulled it from Overture query spam, right? ( I am really asking, since you guys are the experts).

Yet in find all mode, "search engine optimization" is zoomed in on- on the SERP and I am not certain how much rarity plays a role- in this case it must be the word "optimization".

Actually after looking at it I can see how the top 5 queries cluster around the rare keyword and proximity follows- bringing back some pretty wacky spam pages.

Real questions however-

1. What the heck are your average customers and prospects really typing in? Dan has an excellent chapter in his book on specific versus broad terms- how a visitor behaves when they are using the search engines for one or the other.

2. How many people really use artificial awkward keyword queries to make it worth your while to optimize your website or webpage for these phrases?

3. If you do have a lot of awkward keyword phrases being used to arrive on your site, what are the reasons for it, and how can you take advantage of it? Doesn't it make more sense to drill into these terms for higher traffic expressions . . . thus increasing the "rarity" factor for more expressions across your entire website theme?

4. Is it possible that having above average amounts of "expert verbiage" from the brains of subject matter experts about a website 'topic or theme' create more traffic to your website overall?

Fascinating discussion-

Thanks again guys!

- Russell

[DanThies]

I hope we can ask some questions without getting hit by a flying chair.     

We wanted to thank you for the very thorough geekspeak and reduction of the topic of FIND ALL and EXACT MATCH.

It's really a pretty simple question. Michael's confusion is based on mixing up ranking and retrieval, or rather seeing them as the same thing. The reason I replied to him at all, is that he dispenses these flip answers all over the place, with sweeping generalizations that just don't hold up. When the question is "how to deal with awkward phrases," telling a copywriter that the word order doesn't matter, well, that's just not true.

Michael is correct that FINDALL means "find all of the documents where all of the terms occur, regardless of order and proximity." A FINDALL search tells the search engine to retrieve all the documents that include all of the words in the query.

When Michael assumes that "regardless of order and proximity" also applies to the ranking of those retrieved documents for presentation to a searcher, he goes too far.

Even if Michael's assumption were correct, the fact that some percentage of searchers WILL use an exact phrase search to find what they want argues for the opposite of what he advises.

it seems that the original purpose (to help the novice mindset, if not the specific newbie) should come back around.

I agree.

1. What would you tell a newbie about the issue of "awkward phrases"?
Our position has been to encourage them to add natural, context-friendly expressions where it makes sense and the proximity and rarity will (generally) take care of itself. This is GENERALLY what we understood Michael to be saying. 

If ... they get multiple "artificial" queries showing up consistently in their web-log files, they may want to consider using them in some sort of on-page context, but even better they may want to drill into the awkward terms and find more coherent phrases and themes with higher trafficked expressions.

One aspect of SEO copywriting skill is understanding the indexing process. When you see an awkward term like "bowel cancer early symptoms" you can write that into your text very naturally:
Industrialized nations have seen a marked rise in the incidence of bowel cancer. Early symptoms of bowel cancer include...
Now let me remove the punctuation, to translate that into what a search engine will read:
industrialized nations have seen a marked rise in the incidence of bowel cancer early symptoms of bowel cancer include...

have about as much interest in "forced" awkward phrases as I do in the Overture suggestion tool.

With you all the way on both... but as intelligent humans we don't have to write like a screen scraper.

There are a ton of sites that have it on the page as an "exact match" because they pulled it from Overture query spam, right? ( I am really asking, since you guys are the experts).

Overture/Yahoo's search term tool isn't based on the exact terms people type in. Singular and plural will be combined, the words in phrases will be reordered alphabetically or otherwise, etc.

1. What the heck are your average customers and prospects really typing in?
2. How many people really use artificial awkward keyword queries to make it worth your while to optimize your website or webpage for these phrases? 
3. If you do have a lot of awkward keyword phrases being used to arrive on your site, what are the reasons for it, and how can you take advantage of it? Doesn't it make more sense to drill into these terms for higher traffic expressions . . . thus  increasing the "rarity" factor for more expressions across your entire website theme? 

I wouldn't obsess about rarity, and "rarely written" usually translates to "rarely searched." Just remember that the engine can't find your page if the words aren't on it (or in links pointing to it). Increasing the frequency with which you use rare words doesn't sound like a recipe for good copy.

You can find all kinds of different ways that people are searching by watching the incoming traffic. It's even easier if you run broad matches in your pay-per-click campaigns. Running ads with a broad match for bowel cancer would give the original poster a lengthy list of real world search terms.

4. Is it possible that having above average amounts of "expert verbiage" from the brains of subject matter experts about a website 'topic or theme' create more traffic to your website overall?

Maybe, maybe not... or does it just kill a good thread? In other words, it's possible, but speculative... you might bring in more traffic, you might annoy a lot of users too.

[Michael Martinez]

Mike,

Just to be clear, you believe that if:
1) I write two web pages, one using keyword research marketing and another using keyword marketing research in title and text
2) These two pages would be treated, by all search engines, as equally relevant documents for either search query

Is that accurate?

Dan

I don't have time to respond fully, but "all search engines" is clearly inappropriate, as I never said anything about "all search engines".

Furthermore, your first point is extremely vague, so I'll pass on your hypothetical.

[projectphp]

Dan's point is an important one. SEs retrieve a lot of pages for a query, but that doesn't equate to how they rank them. It is like a casting call. A lot of people qualify for the general look, but how these people are ranked is a seperate issue.

The take away is that ou need to first qualify to rank, and then you need to rank. If you miss either of those two (qualifying or ranking), you are screwed.

Furthermore, your first point is extremely vague, so I'll pass on your hypothetical.

Vague? Which "first point" is vague? He asked a pretty specific question, VERY specific I would say, and to respond so flippantly (and confusingly) is poor form.

Honestly, if you don't have time, fair enough, but then why bother writing a two liner like that? Surely waiting and writing something when you do have time would be better, no?

Because surely the only reason you posted wasn't that you aren't one of those people that need the last word no matter what, as you seem a much bigger man than that.

[MichaelM]

I don't want to get stuck into this, Im just quoting Google:

Keep in mind that the order in which the terms are typed will affect the search results.

Source


I know what Google say happens isn't always what actually does happen, but when I ran a few example searches, just changing the order of my keywords (In a findall search), I did get different results each time. Not massively different, but there was some difference.

[projectphp]

Still didn't answer the question!

Just to be clear, you believe that if:
1) I write two web pages, one using keyword research marketing and another using keyword marketing research in title and text
2) These two pages would be treated, by all search engines, as equally relevant documents for either search query

Is that accurate?

[DanThies]

I don't think Mr. Martinez has time for it right now. He's probably busy, I assume the other bloggers @ SEOMoz have to take up the slack while Rand has been out here in San Jose.

[projectphp]

Whoops! My Bad! I saw MichealM and got 'em mixed up.

Appologies from me!