Skip navigation
SEO Website Audit

9 Steps to Diagnosing Lost Search Engine Traffic

June 30, 2010

For the past few weeks I've been tasked with reviewing a few different sites that have seen a loss in traffic – their owners hoped to find out why. I love these kinds of reviews because it's like solving a mystery or figuring out a puzzle. While it's not always possible to determine the exact cause for the traffic loss, I can usually make some educated guesses based on what I dig up in Google Analytics.

Here are 9 steps you can take to diagnose the cause of lost search engine traffic:

  1. Determine what type of traffic loss you're dealing with. Many people look at Google's overview page, see a loss of overall traffic to their website, and assume that they must have lost their rankings in Google and the targeted traffic that comes with it. This may or may not be the case. Be sure to check for search engine traffic, and even more specifically Google non-paid traffic.

  2. Look at the extent of the traffic loss. Your research will be very different depending on whether there was a gradual decline in traffic or a sudden, drastic drop. I reviewed a site last week that lost all of their Google unpaid traffic overnight! This sort of loss is typically a technical issue such as a robots.txt file or a nofollow directive that keeps search engine spiders from indexing your pages. Sometimes it's not actually a loss of traffic at all – your analytics code could have been inadvertently removed from all or most of the pages, making it appear like a traffic loss. I have seen all of the above more times than I can count in just the last couple of months!

  3. Compare apples to apples. Many businesses are cyclical or seasonal. A gift site may see huge spikes in traffic the months leading up to Christmas or the weeks before other holidays. This means that comparing any month to the previous month may not tell you the whole story. A drop in traffic in January is probably fairly normal for a gift site. If you've got more than a year's worth of data, you'll want to compare this month's traffic to the same month in previous years. Ideally, you'd of course want to see a growth in traffic. And if you don't, then you may very well have a problem on your hands. If you don't have data that goes back that far, you can compare month to month, but be sure to take the data with a grain of salt.

  4. Review and filter out "brand" traffic. Most websites get a lot of Google traffic from people who've typed some version of the name of their company as their search query. You'll want to note whether those visitors have significantly increased or decreased. If you receive fewer visitors for your brand, this could be caused by a decrease in marketing and advertising. Once you make note of the brand traffic, you'll want to filter it out so you can study actual keyword traffic, which is what real SEO traffic consists of.

  5. Analyze which keyword phrases have had a significant decrease in visitors. Now that you've filtered out the brand traffic, you should be able to see the keyword phrases that are bringing you the most traffic. If you have lots, you may want to view 100 phrases at a time rather than the default of just 10. Are there lots of keyword phrases that seem to bring far fewer visitors over the last few months as compared to last year at the same time? You may also notice some that are bringing significantly more visitors.

  6. Do a quick Google search for the phrases. If you're not seeing any pages from your site on the first page in Google, it may or may not be a clue (given the fact that everyone sees different search results) but it is definitely a cause to investigate further. If a page from your site does show up fairly high in the list, it could just be that fewer people are searching for that phrase now as compared to before. Or it could be that your listing isn't quite what the searcher is looking for based on your title and descriptive snippet. There might also be other results for the keyword phrase that have images or video embedded whereas yours doesn't. Or there might be local map results showing up that make your result less appealing.

  7. Review the landing page for the keyword phrase that lost traffic. Is there any obvious reason why it's not bringing in as many visitors as it used to? Does it even exist anymore? Did it change substantially at some point during the year? Did it get buried deeper into the site architecture for some reason? Is the content duplicated from other pages within your site or contained on other websites? Were there links pointing to it at some point that no longer are? Does the copy read naturally, or are there a few extra instances of the keyword phrase than really makes sense to a person?

  8. Review your long-tail traffic. Since the end of April and early May 2010 a few large sites lost a substantial amount of traffic for keyword phrases that brought small numbers of visitors individually, but in aggregate they made up a lot of website traffic. You'll want to filter your keywords to those that have only a few visitors (even just 1) and see if there are significantly fewer of those than previously. If this is the case, Google has gone on record stating that they're doing a better job at sending long-tail traffic to more meaningful and relevant pages than they used to. Which means you'll have to go above and beyond what you're currently doing if you want to get that long-tail traffic back.

  9. Decide if you're dealing with a search engine penalty. For drastic drops, in the rare cases where it's not a technical issue, you're most likely dealing with a penalty. You can check your Google Webmaster Tools account to see if there is a notification of a penalty, but they don't usually bother to tell you. Still, search engine penalties are much rarer than people think. In fact, most website owners know what they've done wrong when they have a search engine penalty. There are some cases, however, where they may have been duped by a less than scrupulous "SEO" company. The penalties I've seen seem to occur on sites that have no redeeming value because they have the same products and content that can be found on many other sites (often ones owned by the same company), plus they are deeply entrenched in massive link farms. It's likely that they are also hosting part of the link farm on their own site in the form of a link directory. If this is what you find, you may be better off to start from scratch rather than trying to salvage the penalized domain.

I hope these steps help you diagnose your own loss of traffic. I imagine they will keep you busy for quite some time!

If you feel that you've lost search engine traffic and would like me to take a closer look to determine what the problem might be, please fill out our contact form and mention it in the "Business Goals" section.


Jill Whalen is the CEO of High Rankings, a Boston SEO Consulting Agency.

If you learned from this article, be sure to sign up for the High Rankings Advisor SEO Newsletter so you can be the first to receive similar articles in the future!

Related Post: Why SEO in All the Right Places Doesn't Cut It Anymore
Post Comment

 John Allen said:
Great write up Jill!
I would add:
1) Look to see if there has been any significant change in a referral source.
2) Check to see if website has been off-line due to a server outage or update?
3) Could there be a mechanical problem, such as an incorrectly written NoIndex tag in the robots file?
4) Were any changes made to the website prior to the loss?
 Sean Hughes said:
Hi Jill,

Yes another very interesting post, and one which is very relevant for a client I am currently doing for here in the UK. One thing I wasn't aware of was the "Everybody sees different search results" I have now looked into this for a better understanding, and would advise everybody reading this to get an understanding.
I find that filtering out all the keywords that include the brand name goes a long way in finding out what a particular clients site is currently optimised for, In the case of our clients that is usually just the brand name with a small percentage of long tail keywords left to analyse for volumes and competition for future campaigns.
Again thanks for another insightful interesting post.
 Matthew Diehl said:
Hey Jill - Great daignostics post!

I have to say #4 (brand exclusion) is something that a lot of folks miss when they are reviewing analytics when diagnosing issues.

One other thing that you should check for is a hacked site. Checking the outbound link profile for links that shouldn't be there can help to whittle down the problem as well.
 Warren said:
What if a site has gotten an algorithmic penalty (not been de-indexed)? Is this more often due to on page problems or to improper linking strategies?
 Jill Whalen said:
Warren, it can be either.
 RobjH said:
Hi Jill,

With the new all now not provided, the keywords in Analytics now become more challenging as the only info we now have is via webmaster integration, I have setup a shortcut for non branded key phrases but it does not give you a lot of info, how are you tackling the new loss of keyword data?
 Jill Whalen said:
I've been looking at landing pages instead for quite some time.