Jump to content

  • Log in with Facebook Log in with Twitter Log In with Google      Sign In   
  • Create Account

Subscribe to HRA Now!

 



Are you a Google Analytics enthusiast?

Share and download Custom Google Analytics Reports, dashboards and advanced segments--for FREE! 

 



 

 www.CustomReportSharing.com 

From the folks who brought you High Rankings!



Photo
- - - - -

Such A Big Bump In Pages Indexed - Please Help


  • Please log in to reply
14 replies to this topic

#1 BIGGroup

BIGGroup

    HR 1

  • Members
  • Pip
  • 6 posts

Posted 21 October 2014 - 04:34 PM

We've almost 400k pages had indexed till date that graph shows in GWT for removed . Last week we saw Google has indexed 100K + more pages but in xml sitemap we've included 289,105, moreover, while we visit the SERP(s) we found 700K pages indexed. 
 
How do we measure that which number are true or Google indexed the same pages but with varied query strings (parameters) in the url?  But we fixed all parameters in GWT which shows Representative URL. 
 
How do we measure that out of those xml file numbers which pages Google keep indexing?
 
Awaiting for your reply. 

Edited by chrishirst, 21 October 2014 - 05:13 PM.


#2 chrishirst

chrishirst

    A not so moderate moderator.

  • Moderator
  • 7,718 posts
  • Location:Blackpool UK

Posted 21 October 2014 - 05:17 PM

 

We've almost 400k pages had indexed till date that graph shows in GWT for removed . Last week we saw Google has indexed 100K + more pages but in xml sitemap we've included 289,105, moreover, while we visit the SERP(s) we found 700K pages indexed.

Why does it matter?? Or rather why do you think it matters?

 

 

 

But we fixed all parameters in GWT which shows Representative URL.

'Fixing' it on the server would be more useful.



#3 BIGGroup

BIGGroup

    HR 1

  • Members
  • Pip
  • 6 posts

Posted 21 October 2014 - 09:51 PM

Thanks chrishirst for the reply. 

 

Because it matters that in xml file we included total 289,105 pages, out of those Google has indexed 600k pages, also it shows on SERP listing. So we're suspecting that Google has indexed same pages with query string parameters? How do we find these new pages?

 

Regarding fixing on the server it can't be possible, on the site we need to load the same content from different sub NAV. menu so that's why we used URL Parameter tool to tell Google not to crawl dynamic parameter URL.

 

 

 



#4 chrishirst

chrishirst

    A not so moderate moderator.

  • Moderator
  • 7,718 posts
  • Location:Blackpool UK

Posted 22 October 2014 - 08:02 AM

 

tell Google not to crawl dynamic parameter URL.

You cannot actually tell Google to do or not to do anything using the webmaster tools interface

 

You can suggest what you would prefer to happen, but ultimately Google will decide.

 

A 'sitemap' is only useful in the case of a site navigation creating crawling problems, it is NOT an authoritative statement of the URLs that should be indexed.

 

 

also it shows on SERP listing

No, a "site:" search shows the URLs that google knows about, that does NOT imply or indicate that ANY of those URLs are going to be shown to 'normal' users for a search query.



#5 BIGGroup

BIGGroup

    HR 1

  • Members
  • Pip
  • 6 posts

Posted 22 October 2014 - 08:30 AM

Yes, so how do I address this issue that question is, in xml sitemap included 289,105 pages, but in GWT shows 600K pages. 



#6 chrishirst

chrishirst

    A not so moderate moderator.

  • Moderator
  • 7,718 posts
  • Location:Blackpool UK

Posted 22 October 2014 - 10:17 AM

What is there to address???

 

If you think that Google is crawling parametrised URLs unnecessarily, then block the URL parameters that should not be crawled using robots.txt, other than that, there seems to be nothing to be overly concerned about.



#7 Jill

Jill

    Recovering SEO

  • Admin
  • 33,244 posts

Posted 22 October 2014 - 05:54 PM

Most likely it's a canonical problem. Look up how to properly use rel=canonical and implement it. 



#8 BIGGroup

BIGGroup

    HR 1

  • Members
  • Pip
  • 6 posts

Posted 22 October 2014 - 09:25 PM

Thanks guys for your valid reply.

 

Here's one question: Does Google automatically ignore same query string parameters on the site or we need to mention by using appropriate option like rel=canonical, robots, etc.?

 

For instance, we also have newsletter tagging URL, but Google never indexed those pages.. 



#9 chrishirst

chrishirst

    A not so moderate moderator.

  • Moderator
  • 7,718 posts
  • Location:Blackpool UK

Posted 23 October 2014 - 05:04 AM

 

For instance, we also have newsletter tagging URL, but Google never indexed those pages..

If there are no links to those URLs Google won't know how to find them.



#10 Jill

Jill

    Recovering SEO

  • Admin
  • 33,244 posts

Posted 23 October 2014 - 09:01 AM

Not necessarily true Chrishirst. They find all sorts of stuff that have no links pointing. 



#11 chrishirst

chrishirst

    A not so moderate moderator.

  • Moderator
  • 7,718 posts
  • Location:Blackpool UK

Posted 23 October 2014 - 09:47 AM

So people keep saying, ... I never seen it happen in real life.

 

I have had URLs 'live' for years that Google have never found without a reference to them appearing somewhere else.



#12 Mikl

Mikl

    HR 5

  • Active Members
  • PipPipPipPipPip
  • 345 posts
  • Location:Edinburgh, Scotland

Posted 23 October 2014 - 10:19 AM

Not necessarily true Chrishirst. They find all sorts of stuff that have no links pointing. 

 

I can confirm that.

 

I registered an entirely new domain name, and set up a site with five pages. I didn't tell anyone about it, didn't post any links anywhere, didn't "register" the site in any way, didn't even paste a URL into an email. Unless I mumbled an occasional URL in my sleep, the world was totally ignorant of the site and its contents.

 

And yet, within six days, all five pages were present in Google's index. Don't ask me to explain this. But it did happen.

 

Sorry, this probably doesn't help with the discussion in this thread, but I thought I'd mention it anyway.

 

Mike



#13 chrishirst

chrishirst

    A not so moderate moderator.

  • Moderator
  • 7,718 posts
  • Location:Blackpool UK

Posted 23 October 2014 - 10:27 AM

 

And yet, within six days, all five pages were present in Google's index. Don't ask me to explain this. But it did happen.

 

Look towards your registrar publishing a "recently registered domain names" list.



#14 Michael Martinez

Michael Martinez

    HR 10

  • Active Members
  • PipPipPipPipPipPipPipPipPipPip
  • 5,325 posts
  • Location:Georgia

Posted 23 October 2014 - 06:34 PM

Here is an incomplete list of ways search engines can discover new Websites:
  • The domain was previously registered and had some links pointing to it
  • The domain was previously registered and the search engine simply "remembered" it and came back crawling from time to time
  • Someone puts a blog on the domain and it "pings" (Wordpress does this automatically for the default "Hello World" post)
  • Someone mistypes a link and sends the search engine crawling to your domain (this could even be an old link)
  • You authenticate the site with the Webmaster dashboard (Bing and/or Google)
  • You put AdSense (Google only) on the site
  • You authenticate the site with the Webmaster dashboard and submit it for crawling
  • The domain registration is picked up various automated Websites that publish links to your site


#15 Mikl

Mikl

    HR 5

  • Active Members
  • PipPipPipPipPip
  • 345 posts
  • Location:Edinburgh, Scotland

Posted 24 October 2014 - 05:00 AM

Look towards your registrar publishing a "recently registered domain names" list.


Yes. That's entirely plausible.

Mike




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users

We are now a read-only forum.
 
No new posts or registrations allowed.