14 replies to this topic

[Michael Martinez]

For the past couple of weeks I have been watching Google delist many Web sites that don't violate its guidelines or do anything sneaky. I have seen this happen twice before since February 2005. SE Roundtable now reports that pages are dropping out of Google's index in large numbers, according to a lengthy discussion in another forum.

I have been waiting to see how many people were getting hit before saying much about this data dump.

The last two data dumps (the 2005 February data dump and the Summer 2006 data dump) lasted about 2-3 months. I cannot predict how long this data dump will last.

However, it has been my experience that Google will pick up new content faster than it will reindex older content. While I cannot promise that adding new pages to sites that are losing content in Google's index will help, I don't see why it would hurt as long as the new pages contain unique and useful content.

People should keep in mind that Google has reingineered its search engine twice since February 2005 so we cannot be sure until it's over what the visible characteristics of this data dump will be.

I welcome any comments here, especially anecdotes.

If people are afraid they are being penalized for an unknown reason, I'll be glad to accept PMs for the next week or so where I'll look at your sites and offer private opinions. In fact, I have done this in the past and shared my findings on the basis of aggregate reviews. I won't pitch any services to you and I will maintain your information in the strictest confidence.

I'll stop by each day, as my schedule permits, and watch this thread. I may follow up from time to time as well.

[ioma]

Over the past week or so my site has has increased in number of supplementals (increasingly fewer results returned for site:www.mydomain.com/*)

I've been looking at a couple of forums to see if there was any mention of a google update but found nothing so glad this has been posted by a few people now.

Been getting a good variety of inbound links lately and increasing the content of my existing pages a bit to make them more unique from one another so was kind of expecting a decrease in supplementals!

There are less indexed pages then normal in total also but not by as much of a difference as the increase in supplementals.

Also noticed more shopping comparison sites appearing in search results usually dominated by online stores.

[DanThies]

Michael, are there a significant number of reports from those with fewer than 1000 pages indexed?

The SER story quotes someone who "dropped" from 1700 to 1300, but since you can't confirm the accuracy of either count, I'm not sure we really know if the sky is falling.

If there are a significant number of smaller sites reporting a "data dump," isn't this subject to reporting bias, since nobody complains when Google indexes more pages?

[Michael Martinez]

Dan I can only be sure of what I see for my own network and client networks. I have some sites almost completely in the Supplemental Index that are not losing any pages. I have some sites almost completely in the Main Web Index that have all but vanished.

Most of the sites I monitor have fewer than 50 pages. A few of the sites that are suffering the worst loss of visibility for me actually have pretty good linkage.

Xenite's page counts have gone down from about 1900 to around 1200, give or take. Our traffic has not seen much of a decline, maybe 10-15% (but Google doesn't drive that much traffic to Xenite). Google referrals for Xenite run between 25,000 and 35,000 a month (when you add up all the little places from which they can send traffic).

SF-Fandom, on the other hand, has lost about half its normal traffic. But we get a mixed crowd there, including a lot of young people. Are they simply gearing up for school or did we lose a major source of traffic? Right now Google shows about 27,500 pages from SF-Fandom, which is pretty low. It looks to me like our live forum posts got dumped (although I am only seeing about half our static archive).

Between the two domains, SF-Fandom seems to have lost more value-passing backlinks than Xenite over the past year (it was about a year ago that SE Roundtable published the first report of skunky things about the Supplemental Index).

So it looks to me like this data dump is link-related, and sites that have good, strong linkage should fare better than sites with weak links (but some exceptions occur, as I noted above). Xenite has always had pretty good linkage.

Right now, that's all I can share that I feel confident about.

Edited by Jill, 06 September 2010 - 11:25 AM.

[DanThies]

So how do you know that they really had 1900 pages or 1200 pages or 27500 pages? They will never actually show you more than 1000, and the inaccuracy of the page counts returned with site: has been widely known for years.

[Michael Martinez]

So how do you know that they really had 1900 pages or 1200 pages or 27500 pages? They will never actually show you more than 1000, and the inaccuracy of the page counts returned with site: has been widely known for years.

I know my own site's architecture and content. It's no great task to perform directory-level searches that don't have to return 1,000 or more results.

When you don't see pages in the results that used to be there, you don't have to worry about whether the page counts are accurate.

[DanThies]

Thanks for clearing that up. So are you saying that some sites are losing pages while others are gaining pages indexed, or that the actual size of Google's index is shrinking?

[Michael Martinez]

Thanks for clearing that up. So are you saying that some sites are losing pages while others are gaining pages indexed, or that the actual size of Google's index is shrinking?

I am saying:

• Of some 300+ Web sites that I monitor, many have pages that have dropped out of the Google index over the past couple of weeks
• Other Webmasters are reporting that they have sites which have lost page visibility in the Google index over the past couple of weeks
• Of the pages I control which have vanished from the Google index, most appear to have been in the Supplemental Results index (but most of my pages were there anyway, for a variety of non-alarming reasons)
• I am still getting new content indexed, albeit at a much slower pace than prior to August

The Google index does appear to have dropped a large number of pages from sites that I compete with, but I cannot speak for those Web sites.

My feeling is that this massive data dump may be related to how Google evaluates and processes links. But that's not a thesis I am prepared to defend, justify, rationalize, or explain.

[DanThies]

OK, it's just that you titled the thread with "data dump confirmed," and I thought maybe it was actually confirmed that Google had decided to reduce the number of pages in their index. Which would be pretty unusual, uncharacteristic, and surprising.

Sounds like the news here is that some sites have fewer pages indexed. The first one who noticed it ran off to WMW and posted, causing others to look... those who lost pages say "me too!" and before you know it, it's a "massive data dump."

We aren't going to hear a big outcry from those who actually have more pages indexed, though. Are we?

[Michael Martinez]

OK, it's just that you titled the thread with "data dump confirmed," and I thought maybe it was actually confirmed that Google had decided to reduce the number of pages in their index. Which would be pretty unusual, uncharacteristic, and surprising.

When a lot of people report that they have lost content in the index, it's confirmed that Google is dumping pages. This has happened before and it may very well happen again. They usually recrawl the Web and rebuild the index.

That's why I sub-titled the thread "data dump confirmed" because it has been confirmed through multiple Webmaster reports.

A rumor is where some mysterious other person you cannot find may have said something. We have a trail of reports to follow (including my own).

[DanThies]

Michael... some sites gain pages, some sites lose pages. Sometimes a little, sometimes a lot. This has happened every day, for many years.

[Michael Martinez]

Michael... some sites gain pages, some sites lose pages. Sometimes a little, sometimes a lot. This has happened every day, for many years.

Which has nothing to do with the currently reported activity in the Google index, Dan.

[piskie]

Sometimes, positions become so entrenched that objecivity goes out the window.

[DanThies]

LOL!

[seo.spsoft]

I am saying:

• Of some 300+ Web sites that I monitor, many have pages that have dropped out of the Google index over the past couple of weeks
• Other Webmasters are reporting that they have sites which have lost page visibility in the Google index over the past couple of weeks
• Of the pages I control which have vanished from the Google index, most appear to have been in the Supplemental Results index (but most of my pages were there anyway, for a variety of non-alarming reasons)
• I am still getting new content indexed, albeit at a much slower pace than prior to August

The Google index does appear to have dropped a large number of pages from sites that I compete with, but I cannot speak for those Web sites.

My feeling is that this massive data dump may be related to how Google evaluates and processes links. But that's not a thesis I am prepared to defend, justify, rationalize, or explain.

Yes, thank you for your great information. but why my site was still listed in results. and its cache was no yet taken if i check the cache at google listings it is displaying my page but not by using "cache:site dot com.?????? , please explain