If you don't want to read through this entire post, that's fine - but make sure you understand this before you go: When Google updates (or "dances" - and this is where the term dance comes from) everything varies widely from minute to minute. If your site is gone for a moment, don't panic it may very likely be back (or it may not be, but worrying during the update isn't going to do anything but give you an ulcer).
If you are interested, let's look at the big picture of what is going on. Much of the way Google does updates is relatively new, so a good part of this is going to be speculative as to specifics. The basic gist of it should be pretty accurate, though.
A Brief Modern History of the Google Dance
Up until the middle of 2003, Google did fairly regular monthly updates. The updates would take about 6-10 days depending upon how much new stuff they'd gathered over the course of the month. (I'll explain why it takes this amount of time and what's happening a bit later). It was during this "update" period where newly crawled pages were infused into the database, new PageRank and page linking data were calculated and logged, and other factors were all totaled up and distributed to the various datacenters that serve up Google's results.
In 2002, though, we also found that Google had started to keep a second, smaller database of "fresh" content and it would constantly superimpose the fresh stuff in on top of the older stuff. This content was not actually infused into the main database, but rather placed on top of the rest. At the end of the month, during the regular update, this fresh stuff was then infused into the rest of it.
In late spring (or early summer, I forget which month exactly - and it doesn't matter) of 2003, Google came up with a practical means of doing what I call a "rolling update." The database was constantly updating with a batch of sites (presumably from the same or similar sectors, but maybe not) being updated one day (or two days) and then another batch going the following day, and so on. This meant that there was no "dance" for everyone, but you would still have a short period every month where your site would go nuts - it just wasn't at the same time as everyone else. It'd still have the same updates, PR calculations, and so on.
In December of 2003, we saw our first update in about half a year. This update has been dubbed "Florida" by many. If you see people talking about Florida, know that they are talking about the last 2003 Google update and not a presidential election.
This update really shook some people up. It was huge, but understandably so - and in many cases, it was huge only by comparison to the rolling updates we've been used to. In other cases, though, it was a sign that fire was falling from the sky, the oceans were surging up to swallow entire cities, and that Armageddon was set to hit three days before Christmas.
We've talked extensively about what new technologies and Keep the Faith When the [url=http://searchengineland.com/070621-145956.php]Keep the Faith When the Algo Changes[/url] are (or may be) in effect since Florida, so I won't bore you with them. Let's just move onto the current January update.
Since WebMaster World has decided to put their update discussion behind closed doors (i.e. in the pay site portion of their forum), their tradition of naming the update isn't going to really catch on as well - we have no way of knowing that they've chosen to call it Austin, so I'm going to name this one - the first of 2004 so it needs a letter "A". Because I'm not very creative, I'm just going to go with names or handles of various web professionals that I know. From this point forward, the January 2004 update will be known, by me anyway, as "Update Ammon".
Not only is Mr. Johns high on my list of respected SEO and Web experts, but his name conveniently starts with the letter A. The Mechanics of An Update or Help My Site Is Gone!
It appears that with the new technolgoies Google has introduced into its engine, that we may be starting to see regular monthly updates again - at least for a while. There may be some parts of it that are still done on the rolling basis, and there are surely fresh results getting in there. (I haven't been able to determine if the fresh results are actually put into the database or if they are superimposed, but I'm thinking it's likely still the latter).
Anyway, Update Ammon is much like any other Google Dance. There are wildly fluctuating results depending upon which datacenter you happen to land on when you first type "www.google.com" into your browser. (In the past, we were able to change what we typed in to muscle our way onto the specific datacenter of our chosing and be able to see where things stood. Now, they've taken that ability away from us, so we can never really be sure if we're seeing new results, old results, or the results as they are being shuffled. This makes the whole thing more frustrating for Google Watchers and victims of Egoogle Vision).
Google has several different databases that all interact with each other. There's one that contains all of the text and content of all the pages it has crawled (and another that contains the text of the "fresh" stuff). There's one that contains inherent non search sensitive data about the pages such as PR, and likely things like word counts, the name of the category the parent site of the page is listed in in the DMOZ etc. And, more recently, there seems to be another database that works out semantical relationships between words, phrases, and the pages they appear on.
On the back side, Google has several different datacenters all over the world that store these databases and serve them up to you when you do a search. When you type in www.google.com, you are connected to a datacenter - but you never know which one. It'll try to give you the closest one to you, so it's faster, but if that one is pretty full, it might pick another one for you to connect to. This is why, during an update, you might see old results at one point, then new results, then an hour later, you'll see the old results again - you've switched datacenters. The longer the update has been going on, the more likely you are to be on one that has the new stuff.
So, in order for all of the datacenters to have the same data, the databases must be uploaded to each of the locations around the world. We're talking billions of web page contents, statistics about all of those pages, and now, even relationship data showing how all of these pages fit with other pages on the web. This takes a while.
Most likely, too, the much of the comparative data isn't calculated first and then sent over (it would make transfer time much longer) but rather, the data it needs is sent over, and then that datacenter runs a program that calculates the compartive data once it gets it all. This will create even more fluctuation of the sites on a specific datacenter for a decent period of time. It's during this portion of it where you'll see lots of spammy looking results and pages that just shouldn't be there. The relational math just isn't quite ready but the new pages are already in that datacenter's database.
During the rolling updates, it wasn't as noticeable is it didn't transmit and calculate factors for everything all at once, but amongst various sectors.
Why Does It Have To Be Done This Way? or Why don't they do the damned math and then go live with it?
Part of it is money. The other part is the desire to keep things as fresh as possible. Google could buy a whole bunch more computers at great cost to them and put them in each of the datacenters. But, even if they did that, there would still be the matter of taking the calculated data and transfering it over to the machines that serve it up to us. In essence, they'd be doing the part of it all that takes the most time (transmitting data between machines) twice. That would double the amount of time an update took (even though it would appear to use that things were more stable). And though a local transfer of data is faster than one going across the country or around the world, the sheer amount of data would still take days and not hours to move.
In essence, going this route would double the cost and number of resources needed for Google to work and it'd maybe shave 20% off the time it takes to update. It's just not practical.
Why It's A Bad Idea To React During an Update
It's fine and dandy to speculate on what the changes are going to end up being when an update finally settles down. But you have to remember that it's not likely to be particularly accurate or useful - especially now that we can't tell what datacenter we're looking at and exactly what phase that datacenter might be in (i.e. old results/old math, new results/old math, new results/some new math calculated, but some old math, new results/new math).
Even once every datacenter is showing "new pages" in the index, there is still a period of time where those new results shuffle and settle. At some points you may end up on a datacenter that truly is showing you the way it's going to be - but how do you know?
Always be very careful to not start majorly redoing your pages just because they aren't showing up for a while during the update. You may change them for the worse and the pre-redo pages will begin to rank well and your post-redo ones will start getting crawled for the next update.
During the update, get a cup of coffee and start exploring how to optimize for INK or whatever you think Yahoo is going to be using in the coming months. Do anything but try to second guess what Google may be doing.
Don't Forget About the Post Update Tweaks
Okay, the update has settled down and it's over. Or is it?
Some sectors are chock full of spam. Some sectors are bringing up some pretty weird results.
Don't worry. Google's on it. They have people there who run searches and find terms that are bringing up fishy results. Those results are then sent over to the "algo master" who has a look at them.
Google does not (according to them) mess with results on a term by term basis. In other words, if your pet term is looking spammy, they aren't going to go through and tag certain pages as "bad" so they vanish. (Those pages may be what someone is looking for with another term, who knows?)
What they will do though, is go into their Almighty Google Algo Control Kit (AGACK) and start tweaking some of the thousands of variables and constants that are there in an effort to improve things overall. What fixes one problem in one sector will likely fix another problem in another sector leaving the sectors that were unaffected still unaffected (because those variables never came into play very much in the first place).
Conclusion
If you have been having pretty good ranking in Google and you vanish during an update (and the update is still on) don't panic. It may not be a good sign, but it may not mean anything.
If, after the update has completed, you are still nowhere to be found, don't panic, the tweaks of the algo may bring you back at least most of the way to good. You are going to probably want to do a little work, but again - don't panic.
If, once the tweaks are done you still are missing - don't panic. You've got some work to do, but it's most likely not a call to do a total rework of your SEM/SEO strategy - unless, of course, you were spamming or otherwise misleading the SE's in some way to obtain your results.
Let it be said that if your page was doing fine before the update and now it's gone and you've used decent and non misrepresentational nor misleading techniques, you definitely don't want to panic. Chances are, you were focused on one area of SEO that was working well, but something in the algo changed to reduce the importance of that area you were focused on. Diversify your efforts, do not redo them. The more factors you employ to rank your page well, the less likely any one algo change or update will seriously effect you. Even if you are using some of the more "muscular" SEO techniques, you're far less likely to have a sudden disappearance if you are spreading your eggs out over many baskets.
Be diverse and constant in your efforts (regardless of update schedule) and never panic yourself into a total revamp of your site and optimization techniques (whatever those techniques may be) and soon, you will find that you aren't vanishing from those results as frequently during an update. You may move down a bit and have to improve a bit, but you should never (well, almost never, anyway) have to scramble to try to get yourself back from oblivion.
---
I hope this was helpful in some way. If you feel like you have read this before, you might have - I'm posting a copy at my home at Cre8asite Forums as well as sharing it with my friends over at High Rankings.
Enjoy the rest of Update Ammon and stay tuned for Update Bragadoccio!
G.
