Jump to content

  • Log in with Facebook Log in with Twitter Log In with Google      Sign In   
  • Create Account

Subscribe to HRA Now!

 



Are you a Google Analytics enthusiast?

Share and download Custom Google Analytics Reports, dashboards and advanced segments--for FREE! 

 



 

 www.CustomReportSharing.com 

From the folks who brought you High Rankings!



Photo

Google Webmaster Tools


  • Please log in to reply
7 replies to this topic

#1 H2OSurfer

H2OSurfer

    HR 1

  • Members
  • Pip
  • 4 posts

Posted 01 January 2016 - 02:02 PM

Over the Past 8+ Months we have lost almost 39% of our page in Google. That is right De-indexed.

 

We have been online since 2009.

 

I have gone over to Google Forum and tell you exactly nothing on What the problem can be.

 

We have no Known Server Errors

One DNS error in the past 4years

No Sitemap Errors

No HTML Errors other then short meta descriptions on 42know products

No New Manual Actions (reconsideration request from a site owner and was removed 6/6/2014)

There were crawl errors after We made the site Mobile Friendly, But they we 404 pages. They have been fixed with 301

0 Pages with blocked resources
No manual web-spam actions found.
 
What gives?
 
I have tried to reach out to google but there is no a single person you can contact directly. They refer you to the Forum.
 
We have nearly 190,000 products online and 39% of them are not being indexed
 

 

 

Attached Thumbnails

  • google-index2.png


#2 chrishirst

chrishirst

    A not so moderate moderator.

  • Moderator
  • 7,718 posts
  • Location:Blackpool UK

Posted 01 January 2016 - 07:06 PM

 

 

We have nearly 190,000 products online and 39% of them are not being indexed

 

And how many of the 39% are close duplicates ("boiler plate" text) of the other 61%?



#3 H2OSurfer

H2OSurfer

    HR 1

  • Members
  • Pip
  • 4 posts

Posted 01 January 2016 - 09:11 PM

Thats a great questions.

 

My First Site Map Consist of just about 425 categories but only 412 were indexed. There is no tool that I have been able to find that monitors the exact URL and checks it against what has been indexed.

I spent literately 1 Full Day tracking each link. to Find Which one was not indexed.

 

Technically none of the links are duplicates. There maybe some that are close but that shouldn't really make a difference. Even if duplicate it still should at least be indexed.

 

Since we deal with electronics so I wanted to compare

 

I understand duplicate content which we are working on to minimize. But..
Sites like
Amazon 156million pages indexed in google  
Digi-key 19million pages indexed in google  
Ebay 19million pages indexed in google  
Allied Electronic 6million pages indexed in google  
Newark Electronics 3milion pages indexed in google  
 
Each of these site have a similar setup like ours who also sell multiple like products with content that is similar or even duplicate
 
In addition in Google WMT
 
Duplicate meta descriptions     15
Long meta descriptions     1
Short meta descriptions     42
Duplicate title tags     2

Non-indexable content     Pages
We didn't detect any issues with non-indexable content on your site.  <<<<<<----------- See????? WHat Gives

Edited by Jill, 02 January 2016 - 09:33 AM.


#4 qwerty

qwerty

    HR 10

  • Moderator
  • 8,695 posts
  • Location:Somerville, MA

Posted 02 January 2016 - 08:28 AM

Comparing your situation to Amazon's isn't going to be helpful. That site has millions of backlinks, which gives them rather high PR (Google may not share even an inaccurate version of that metric with us anymore, but it's still there) and that has an affect on crawling and indexation.

 

It's possible that adding structured data for your products actually helped Google to determine that a lot of your URLs didn't need to be indexed, and that might actually be a good thing. The real question is whether this affected your organic traffic. Is traffic to the pages that remain in the index up or down since this happened?

 

I'm not suggesting your hire him, but look up Alan Bleiweiss. He does audits on very large sites (many millions of pages, in many cases), and he often posts about recommending the deletion or de-indexation of large numbers of pages that tend to waste the sites' crawl budget, and seeing big bumps in traffic once those pages are out of the way. It's not always about making sure everything's indexed. It's more about making sure everything unique, useful, and likely to convert is indexed.



#5 H2OSurfer

H2OSurfer

    HR 1

  • Members
  • Pip
  • 4 posts

Posted 02 January 2016 - 09:57 AM

Ok the structure data was implemented on Aug 24 2015 along with Mobil friendly site. The website is device responsive so it reconfigure itself based on device and screen size.

 

I would also add that our direct competitors an NOT Mobil friendly and they do not use structured data.

 

As you can see from the attachments above the de-indexing began prior to 8/24

 

There is no way I can compare our site to Amazon. What I was suggesting is that they have 1,000's of products similar in description, makes and models. Yet they don't seem to be De-indexed.

 

as far as traffic see attachments

 

 

I tried to over lay the De-indexed pages with Google Analytics

 

google-index3.png

 

There is a direct relations between the two.

 

 



#6 chrishirst

chrishirst

    A not so moderate moderator.

  • Moderator
  • 7,718 posts
  • Location:Blackpool UK

Posted 02 January 2016 - 10:58 AM

 

 

Ok the structure data was implemented on Aug 24 2015

 

So, .... What do you expect?

 

You changed the entire SITE URLs .... .... Of course there is going to be a direct correlation. As you stated the original URLs were returning a 404 status, which was eventually 'fixed', possibly by which time Google's system had already marked them as missing and reduced the crawl schedule, so now it IS going to take even longer that the current period of a few months later to fix the indexing. EVERY SINGLE link both external  and internal if you consider those to be separate,  that 'points' to your site URLs now has to be re-evaluated with the redirect in place. Search engines store the URLs of every link that has ever found and when you change your URLs, ... they have to play 'catch-up', possibly for years to replace/update all those references to pages that once existed at point 'a', but are now [hopefully] found at point 'x'.

 

Sure by all means, compare your site with Amazon or Ebay and the fact that they have 'X' number of URLs indexed, but ALSO take into account that neither of those sites have been completely 're-structured' in the last FIVE years never mind FIVE MONTHS ago. Think back to how long it took from the site launch to when most of the site URLs were 'indexed', and consider that, as a 'best cast scenario' it may well take around one third of that time to reach a similar status, maybe a bit longer depending upon how or if the 404 status caused some URLs to be removed from the crawler schedules. Worst case scenario is that it never ever will fully recover unless you market ALL the pages and products that are 'important' or 'valuable' (in terms of sales of profitability) to you and your 'site'.



#7 H2OSurfer

H2OSurfer

    HR 1

  • Members
  • Pip
  • 4 posts

Posted 02 January 2016 - 11:10 AM

No There was no URL Restructuring. The 404 were products that have been removed over the past 2 years.

These were items we no-longer support or they were made obsolete

 

I created 301 redirect for all of them.

 

The URL structure is the same as prior to 8/24/15 and they are the same since the beginning back in 2009.

 

I would never change the structure on the mass like that.



#8 chrishirst

chrishirst

    A not so moderate moderator.

  • Moderator
  • 7,718 posts
  • Location:Blackpool UK

Posted 02 January 2016 - 03:28 PM

Okay let's go a bit further while we are squeezing out bits of information.

 

Products that have been removed SHOULD result in a 404 response (or a 410 response if the removal is permanent).

 

If there is a replacement or equivalent product, then a redirect, with a 301 or 302 response depending on whether the replacement is going to be permanent or temporary is warranted and should not cause indexing or crawling issues, other than that, a redirect to one URL, can be seen as disingenuous by users and depending WHERE the redirect leads to could be treated as a "soft 404" by search engines, which is starting to seem as if that is the case.

You do NOT need to redirect every product nor should you, you only need to redirect products that are active. (Active meaning that these pages are being requested by real users ('bot hits and referrer spam does not count as 'active', and deciding what to redirect is not an "I can so I will" operation which should only apply for a site move between domain names or hosting technology. As you may have discovered mass redirects for the wrong problem is (to mix a few metaphors) a whole new kettle of different coloured worms that is likely to turn around and bite you.

 

So let's have some history in a bit more detail, preferably using the same parlance. Around these parts 'restructuring' means "taken apart and rebuilt" whereas 'pruning' dead or old products and pages is just 'house keeping'.

 

Don't need site names, product names or URLs because publishing them here is likely to add to your woes.






0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users

We are now a read-only forum.
 
No new posts or registrations allowed.