Are you a Google Analytics enthusiast?
Share and download Custom Google Analytics Reports, dashboards and advanced segments--for FREE!
www.CustomReportSharing.com
From the folks who brought you High Rankings!
More SEO Content

Does Faster Spidering = More Indexing?
#1
Posted 02 September 2011 - 04:49 AM
I personally don't believe this. At 87k pages a day Google has crawled our entire site in 2 weeks so they should have all of our pages in their DB by now(it's been months since we launched them) and I think they are not index because they are poorly auto spun pages. Am I correct? Would speeding up the pages make Google crawl them faster and thus get more pages indexed?
#2
Posted 02 September 2011 - 06:58 AM
#3
Posted 02 September 2011 - 12:42 PM
Google's crawling and indexing are driven in large part by the internal PageRank they calculate (this is NOT the number you see in the Google Toolbar). But Google engineers have occasionally disclosed they may choose not to crawl parts of a Website that they deem to be of low quality or unusual to visitors.
#4
Posted 02 September 2011 - 08:01 PM
"unusual to visitors"
It is probably very obvious, but I can't quite put a definitive meaning to it.
#5
Posted 04 September 2011 - 02:01 PM
#6
Posted 05 September 2011 - 12:37 PM
However it sounds like I would never be tempted to do nor accept instruction to produce that type of crud.
#7
Posted 07 September 2011 - 02:36 AM
#8
Posted 09 September 2011 - 02:20 PM
My gut tells me that a faster site may see ranking faster updates following optimization, and that new content added 2-3 layers down in the site hierarchy will see faster indexing. I know Google and Bing don't recrawl the entire site every time, but they have to find the new stuff somehow, even if external links don't yet exist.
Plus, it seems likely that site speed is a factor in Google's "quality" calculations.
#9
Posted 09 September 2011 - 02:35 PM
#10
Posted 09 September 2011 - 04:31 PM
Where the content is placed in the hierarchy usually doesn't matter once the search engine has a record of the location. It can schedule new visits based on PageRank-like value, finding new links pointing to the content, or both.
I think the most important crawl efficiency is managed through the internal navigation. The pages that SHOULD be crawled the most are the pages that update their internal links the most. On many sites that includes the home page but on some sites the home page is considered sacred ground and new links are rarely posted there. A LOT of crawl inefficiency channels the crawlers toward the home page without giving them a real reason to go back there every hour or whenever.
This is one reason why so many blogs do so well in search engines. As they push older content down and ultimately off the front page, new links are discovered and crawled and content is indexed more quickly.
0 user(s) are reading this topic
0 members, 0 guests, 0 anonymous users