Why Google Won't Be The Best Search Engine Forever, Google Achilles' heel Options

[IdanDone]

The quick answer is because Google is a text-focusing search engine and most of the internet users don't search for text.

I don't believe people will search for keywords forever.
Today, someone who types SEO and someone who types "search engine optimization" will get a completely different results pages.
In most cases the two looked for the same thing, but Google doesn't care what people want to find, Google deals with the question which keywords people want to find?

Another example,
What happen when I will type "free store" in Google?
I'll tell you what happen, I'll get a 52,000,000 results, but results of what? "Free store" have no meaning in the real world (except there is a store or a website which is called "free shop").
In the real world man who search for free store should explain the predication: Free store of what.

Google recently launches the "suggestion feature" which is a auto-complete box. It may solve some of the problems.
However, I believe the way to a human oriented search engine has not found yet.
SEO is a symptom of the disadvantage of Google, because today SEO experts promote websites - Most of them don't improve the website at all, only the way Google crawler see it.

Before Jill will ask what is my point, I'll write it:

As long people deal with SEO and websites promotion, they promote one thing: The end of the current mathematical engines, as we know it.
Google understands now more than ever, that people won't keep searching keywords forever. More people have learned how to manipulating the search engine and Google knows that and decreasing the importance of technical factors, such as meta keywords and even the importance of text anchor (not inbound links).
The new search engine will be social-Media and human-feedback oriented and thus the new SEO.
If you're a SEO, Don't focus Google algorithm - The search engine of tomorrow (and I pretty sure it won't be Google) won't be a text-crawler based.

This post has been edited by IdanDone: Sep 1 2008, 11:54 AM

[Randy]

but Google doesn't care what people want to find, Google deals with the question which keywords people want to find?

I can confidently disagree with this blanket statement.

For at least 6 or 7 years everyone I talk to who works at Google or those who write about Google describe this exact type of situation as the absolutely biggest thing on their To Do list. They truly do want to try to understand context for more general terms, then deliver better search results. It's the main reason they've spent literally 10's or maybe even 100's of millions over the past several years trying to develop systems to help them better understand what a searcher might be looking for, even if the searcher asks for something very general. (See search personalization, geo-targeting, suggest, etc.)

IMHO it's one of those almost unobtainable goals all of the search engines are striving to accomplish. I feel for them, having an inkling of how difficult the problem is with the massive amounts of Noise for which they must account. I and everyone else who does any conversion testing has to deal with noise, and it can be quite difficult. Even with much smaller noise effects I have to deal with, it's still a critical part of every test I run.

For Google etal it's an infinitely larger problem. Especially when one tries to do it all in an automated fashion like Big G does. If they manage to pull it off I rather doubt anybody will catch them in the next 20 years.

[Randy]

The search engine of tomorrow (and I pretty sure it won't be Google) won't be a text-crawler based.

And what do you think the next generation search engine will do, if not crawl text?

Especially considering the fact that a very large majority of the Web today is made of up text. Is the next generation engine going to simply ignore all of this text? Or will they do something else with it?

I'd love to hear your thoughts and predictions for the next gen search engine.

[Jill]

The new search engine will be social-Media and human-feedback oriented and thus the new SEO.

And just how do you envision people will do online searches without using keywords?

They will searching using....??? What exactly???

[nethy]

Well I agree that Google will not last forever. But just because Nothing lasts forever (especially online), not some inherent flaw in Google. Like Randy said, I think that they're working on being able to understand questions & answers very hard. Harder then anyone, probably. At the moment, indexing text is the best there is.

I'd love to hear your thoughts and predictions for the next gen. search engine.

Always presumptuous, that kind of thing. But I think some major improvements wil probably be made with another step in the general direction that Pagerank was:
I think a potential key might be in understanding some underlying meta information & how different types of information relate to each-other. For example, comments tell you a lot about a particular piece of information. Their content, lengths, volume, etc. So do ads, responses etc. It's a similar sort of relationship that a link is (even though some of them are links). This information is a lot more complex then just link relationships. Just identifying what is a response, comment, ad etc. & what it's responding to, commenting on, advertising, etc. is pretty hard. But I think it's possible. Then you might have commentrank or commenterrank that works in the kind of way pagerank does. Maybe it works in a different way. Comments might contribute to pagerank. Pagerank might contribute to commentrank.

BTW I'd say Google probably has things like this going on. But we know from SEO experience, that 'pages' & 'links' are still probably the central components. But Google have more engines then web search. That's just the one of most interest. I use, image, video, news, blogs & maps search regularly as a user. If I was going to speculate on the components of future ranking algorithms, I'd be looking at these for clues.

[Randy]

Now there's something we debate. Thanks nethy. Somehow I knew you'd come through with semi-concrete ideas. (IMG:style_emoticons/default/giggle.gif)

FWIW, I agree with most everything you said. Though I'm not sure Google will ever make the same mistake of giving a major part of their algorithm a name again. They don't need the publicity like they did back in the early PageRank days anymore. And I think they've probably realized by now that giving a component its own identity makes it a target.

I also agree they're looking at lots of different kinds of searches. And probably going to look at more information that surrounds a link, rather than just that a link occurs and the fact that there's anchor text there. Some might say this will inevitably lead to a more Thematic approach to sites and search. And I guess it does to a certain degree.

FTR, when I start letting myself daydream about how Google might go about figuring out the relationship between one page and another, including all of the elements involved and trying to come up with a statistical model they might be able to develop and tap into to provide better results, I always come back to imagining a meeting between Google's top engineers, with Brin and Page sitting in or course. This brainstorming session from a couple of years ago always goes something like this in my brain.

Brin or Page: So our hardest thing to do is to understand what an individual person is looking for when they type something into a search box. How can be get better at this?

Engineer 1: I think we should offer some type of Geo-targeting, since we already have this on one level with our different search results for different countries. It's a natural lead in.

Engineer 2: I agree, but we need to do more. Maybe some type of Personalization. Something that takes the same basic concept, but expanded so that everybody can use it.

Engineer 3: That's doable, as long as we can get people to go along with it. Do we do it as something people need to sign up for? Or do we offer Personalization as a default?

Brin: Rememeber our motto, Do No Evil. We'd probably need to start it out as being a Labs project where people opted in. Then once it's developed a bit we can make it a default thing. Maybe first expanding it to Toolbar users, since we can identify those folks fairly simply and they've already opted in to sharing data because of the For Enteretainment Purposes PageRank indicator there. Later we can make it cookie based, since we can always give each Google user a unique identifier or even a Group identifier and save it to a cookie. Then it could make it the Default action.

Page: There are also other tools we can utilize to get a better idea of what people might be looking for when they type something into search. Perhaps like a Suggest feature or a Did You Mean feature?

The whole room nods in agreement.

Brin: Okay, so that gets us started on figuring out at a higher level what users mean when they type something into search. However we're still missing half of the equation. We need to better figure out what pages are actually about, how the content and links work together and how real people determine the relevance between them. We need some way to start recognizing the Relationship between page content, links and what people click on to get where they want to go. This is a complete void for us right now. We simply don't have access to this type of detailed data about any pages out there, except those pages on our own sites of course. Any ideas?

Engineer 4: I don't think we need to reinvent the wheel. What you're talking about is nothing more than Analytics. Lord knows we have some money to splash around with between our Adwords income and the IPO, so why can't we simply buy one of the better Web Analytics packages out there and re-purpose it so that Webmasters will use it, while at the same time sending us mountains of data?

Engineer 2: Easy enough to do, but how do we get Webmasters to freely send us all of this data?

Engineer 4: You just answered the question in your question. We make this new analytics package 100% free for people to use, and make sure it's a Best Of Breed type of application. Better than any other free analytics tool out there.

Engineer 1: Interesting idea. We could give it some pretty cool improvements too. We could give Webmasters a way to compare their site to other sites as part of this. Sort of a Benchmarking thing.

Engineer 3: And we could build or buy a tool that gives Webmasters a way to perform conversion testing, something that ties back in with the general analytics tool. Then teach people how to use it.

Brin: Good ideas everyone. How are we going to get the Webmasters and SEOs on board with this?

Engineer 4: They'll follow along. Or enough will in the beginning to get things started. They always do as long as we simply make the tools available for free and don't mention that we'll be collecting and using the data, in a more far reaching way. It'll take awhile to obtain enough data to reach validity, but you're still taking a relatively small percentage of users and Webmasters to get some really valuable data.

Page: Okay, so there's our 5 year plan. Start implementing Personalization to some degree, along with Geo-targeting, Suggest, Did You Mean and any other user-oriented tools we can develop to take care of the User side of things. Identify and purchase a good analytics package we can re-purpose to make available to Webmasters so that we can begin collecting site data. Along with some type of Benchmarking program and a site sales/conversion software suite that allows us to collect data. Making it all free, of course.

Brin: Sounds good. This could really change the way Search is performed by 2010.

This post has been edited by Randy: Sep 2 2008, 07:13 AM

[torka]

I can imagine a similar conversation, Randy, but in my version the Googlers say "Dude!" a whole lot more... (IMG:style_emoticons/default/giggle.gif)

Sorry, couldn't resist... (IMG:style_emoticons/default/whistling.gif)

--Torka (IMG:style_emoticons/default/mf_prop.gif)

[IdanDone]

Hi Randy and Nethi,

You wrote very interesting things.
Randy imaginary conversation is really realistic.

I believe search engines can't estimate quality and usability. It's seems they do a pretty good job when they estimate keywords relevancy, but not subjective factors, such as friendly interface.

The only element who can estimate quality and usability and all other "behind the text" factors is human being - The users.
In general, I believe users should rank websites and their opinions should be compared with the search engines algorithm and other users opinions.

In order to do so the new search engine should find a way to create and track a reliable user so his opinion about a website will be reliable.
Then the new search engine should find a way to rank users reliability. If today the search engine is ranking websites, tomorrow it first of all will rank users reliability and then compare the websites' opinions of that users to the website ranking (the current text algorithm).
In addition the new search engine should find a way to encourage users to rank websites in various factors.

What do you think?

[Randy]

I believe search engines can't estimate quality and usability.

Estimate is the wrong word to use here, though they can and do have ways to measure both usability of their user interface and SERP quality. Google has been at the forefront of this effort, as usual. They're conducting UI testing constantly. At least 3 or 4 times per year over the last several years. And they also measure quality quite often. If you pay really close attention to their code you can see it sometimes, where they're capturing which sites are being clicked upon from their SERPs.

In general, I believe users should rank websites and their opinions should be compared with the search engines algorithm and other users opinions.

Google is doing this to some degree already. Though they're taking it farther than you hint. This is what the whole Personalization ideal is about. Rather than simply gathering click data and amalgamating it across every user profile, they're doing it so that different user profiles will show different sites in different positions. It's one of many reasons why rankings aren't as important as they once might have been, because different users are increasingly seeing different rankings.

What Personalization means for webmasters is interesting to consider. Not only with rankings not matter as much, but when a webmaster is checking rankings the chances that this webmasters Personalization profile matches that of their target audience is basically nil. So if a webmaster gets their site ranking terrific for their profile it may not rank at all for their target audience profile, and vice versa.

The long and short of it is the Users and what they think are becoming more and more important each and every week to search engines, Google especially. People who ignore this trend do so at their own peril.

[Nueromancer]

And just how do you envision people will do online searches without using keywords?

They will searching using....??? What exactly???

Ailien Space Bats will Create a search engine that is Hard Ai and telepathic :-)

for theose of you not familar with the ASB's from soc.history.what-if faq's

"For example, “Well, Alien Space Bats could land the German army in Wales.” The use of the term Alien Space Bats has been expanded to include handwaving difficulties in order to get to an interesting discussion."

[Scottie]

And just how do you envision people will do online searches without using keywords?

They will searching using....??? What exactly???

Vulcan mind meld technology, of course. Duh...

[nethy]

Well done Randy.

I think the analytics thing is possibly on track (but maybe not). Remember that its really a service for adwords advertisers. Making it available to everyone, isn't very expensive, & probably increases the likelihood of advertising with adwords in the future. More importantly, when you raise your conversion rates, you can raise your bid. When everyone else raises their conversion rates, everyone else can raise their bid. When everyone else raises their bids, you have to. The incentive there is probably enough to warrant the Urchin acquisition (google buys a lot of companies. Urchin is just another one, really).

I'm not saying you're not right about the incentive, just that you don't have to be. Other justifications exist. Could also be that one or the other was more of an add on. But the more I think of it, the more I see how Google Analytics was a brilliant move. Either way, it's about the cash cow Search/Search-Advertising (as opposed to all those huge acquisitions of toothpaste companies). Anyone know how much it cost?

Anyway, it's safe to assume that they're at least looking at the possibility of using GA data to improve search. I think there's another question here. Are they:
1 - Looking at the actual analytics of a site to learn how people use it & relate to it learning about each site with GA installed.
2 - Looking at aggregate data & learingin about how people use site in general. Being able to find some criteria for usability, quality, stickiness, etc. & figuring out some some principles for matching page to search by looking at bounce rates etc. & trying to figure out some patterns that distinguish between user A who search for X & went to page Q & spent all day there & user B who searched for X & went to page Q & bounced.
3 - They do some serious Google Science that allows them to come up with unsayable, incoherent, algorithmic systems for evaluating pages.

I think 1 is unlikely & 3 is very interesting.

All that said, we are still only fretting over a small piece of that puzzle. The same principles still basically apply to web search that always did. & they're pretty simple.
titles, headings, keyword in prominent places on the page, anchor text pointing to the page, description & links (pretty much following the PR algo)

I'm sure there is a huge amount of subtlety & innovation at the margins, but going by that 1 line of guidelines, you get pretty far.

Try this expirement:
Take 10 random pages from Google's top 100 results: Without doing a search, put them in rank order for a certain keyword. I think most SEO's would do ok with this exercise by just looking at those few things. So all those subtleties, don't amount to much. yet.

BTW Randy, semi-concrete ?

[Randy]

BTW Randy, semi-concrete ?

(IMG:style_emoticons/default/hysterical.gif) I started out with Concrete nethy, but didn't want to paint you into a corner since all of this discussion is incredibly subjective. So I wimped out and gave you an easy out. (IMG:style_emoticons/default/wink1.gif)

I'm not saying you're not right about the incentive, just that you don't have to be. Other justifications exist.

Completely agree. There are lots of other justifications. And the cash cow of Adwords/Adsense cannot be discounted in the least. It is the cash cow after all, and the justifications are exactly as you laid out. On both the advertiser and content provider sides of the fence.

I think 1 is unlikely & 3 is very interesting.

My best guess? 2, with a strong development toward 3.

You're right on the important factors. Those same factors have been the same since the mid-90's, with link anchor text having a bit less of an effect before Google came on the scene. Still, the same factors have been the important ones for over a decade. Which is I guess why I have such trouble with people continuing to listen to the Theory Of The Week. Can't they see that the algo's really haven't changed all that much? It's just that there's more competition each hour/day/year, so you have to do a better job with the basics if you want to rank well? I don't get it. This isn't rocket science.

Either way, it's about the cash cow Search/Search-Advertising (as opposed to all those huge acquisitions of toothpaste companies). Anyone know how much it cost?

I've never seen an official admission of the purchase price. John Battelle was one of the first to blog about it. He put the purchase price in the $30 million range. My sources both at Google and Urchin agreed with this round figure at the time, though they couldn't confirm for legal reasons.

If it was anywhere near this $30 million figure Urchin was a heck of a bargain.

[nethy]

If it was anywhere near this $30 million figure Urchin was a heck of a bargain.

Absolutely nothing compared to other ones.

BTW: Google buys a lot of companies Interesting to see what they're willing to pay for. More then I expect have to do with advertising. Fewer then I expect have to do with search. There have been 2 more analytics acquisitions since Urchin.

I wonder how much (if any) of these are motivated by competition neutralising (see video stuff) as opposed to technology acquisition.

[IdanDone]

Hi Randy,

Google is doing this to some degree already. Though they're taking it farther than you hint. This is what the whole Personalization ideal is about

You are talking about localization and personalization. It's important issue, but I'm regarding to the reducing of the SE algorithm importance in favor of human feedback on websites, as you can see in Youtube or Digg.

Check out that link: www.techcrunch.com/2008/07/16/google-continues-to-test-a-search-interface-that-looks-more-like-digg-every-day/ Google is trying to check something in that direction, but because the basics of Google is mathmetical and algorithmic, they will never dare to change their search engine basics and become a search engine which is human feedback based.

What I'm saying is human should determine the ranking of the website with consideration other objective factors such as smart algorithms and statistics calculations