Are you a Google Analytics enthusiast?
More SEO Content
Google Super Fast Page Indexing.
Posted 04 October 2004 - 07:03 AM
Recently Google has been running around like a banshee on All Hallows Eve, grabbing as many pages as possible. Yesterday I noticed something that was weird, well a bit out of the ordinary anyhow. Google hit my rugby forum at 4.30am GMT sunday 3 Oct (yesterday) a few hours later that day, Google had updated the cache, I am talking less than 12 hours, actually 9 hours from index to cache (it might well have been sooner, but I only noticed it at lunchtime).
Why is google doing this? why is google running around, hitting a site, updating the cache, and also sending the mozilla based spider along an hour or so later to the same site? I wonder if G is looking for cloaking or any user agent delivery pages?
As an aside, 8 hours from spider to index, and having bots running around like mad things is no way for a broken search engine to behave. It should sit down and die gracefully
Any one else noticed this, or have any comments?
Posted 04 October 2004 - 07:43 AM
In the past it was common to have some new pages quickly spidered and added, but then they'd be out again within a few days, and remain out sometimes for weeks.
This isn't something I look at very often, so if you have data on this, it would be interesting.
Posted 04 October 2004 - 07:52 AM
any suggestion ...
Posted 04 October 2004 - 08:00 AM
One thing I did notice is that it is a php site, so it is index.php I put in place a news page with a load of rss feeds about rugby so that as people hit the site, they would get the latest rugby news to debate. This is of course index.htm. Now this page was not showing in an inurl: or site: search this morning or yesterday. Now the page is showing, but only under the domain name, which would have been the anchor text used on a couple of sites. www.domain.co.uk it is now showing as a partial index.
Posted 05 October 2004 - 08:52 AM
Isn't it related to this: www.axandra.com/news/newsletter128.htm#facts?
"Many webmasters have noticed that there are now two different Google spiders that index their web pages. At least one of them is performing a complete site scan"
I haven't seen any discussion on this, though it seems important, at least to me.
Posted 05 October 2004 - 08:58 AM
There is a thrid option as well though. Google could be spidering deeply to try and find as many links as possible in order to build a massive link matrix to refer to so it can attack link spamming. Google could be building a database of pages, and although the spider is running through the pages, it might only really be interested in links. Once it has done this, it can run its link spam algo, and zap all the link spammers.
Now where is the tin foil for my head
Posted 05 October 2004 - 11:35 AM
The answer is, we don't know then ...do we
Sorry SC what is the question we don't know the answer to?
Posted 05 October 2004 - 12:47 PM
I noticed this as well. We put some new sale pages up in our ecomm site, and they were indexed by google in about 48-72 hours. Made the boss quite happy, to say the least. I noticed this same thing much earlier in the year, when some new pages we put up on a Saturday were indexed by Monday, I was shocked to say the least. Google has been good to us, Yahoo on the other hand, can take months and months to get our new pages indexed.
P.S. I just checked Google and the sale pages are still showing up. These pages were added early last week.
Posted 05 October 2004 - 12:57 PM
Posted 05 October 2004 - 04:11 PM
I don't understand where that statement fits in. Maybe it's just wishful thinking?
Here's a hypothosis for you to chew on:
You said you are using RSS feeds on your sites. Are your feeds from news sources?
If you look at google news, those pages are updated much quicker. The news section is on a different update schedule. Maybe Google decided that because you had a large percentage of news content to treat the page as a news page.
That would cause immediate inclusion.
If this is correct, the question is - "What is it about the page that caused it to be seen as a news page?"
Figure that out and then duplicate the result. Then tell me how you did it!
Here's a summary: Multiple news feeds with links to accepted news sources (accepted by google) may cause a page to be seen as a news aggregator and be indexed on the news schedule rather than the slower normal schedule.
Posted 05 October 2004 - 04:59 PM
The response I gave about there being a 'third' option was regarding the posted link where they discuss there being two options. And yes it is a bit of wishful thinking, I for one am sick to death of playing black dog blacker dog with regard link building.
Thanks for your hypothesis, but it does not apply for the following reasons.
The site is a forum on phpbb, so the index page of the forum is index.php . There is also an index.html page (being .htm it is shown as the default home page), and it is this page alone which has some feedroll newsfeeds on it. This page however, the news page is not indexed, it is just partially indexed. It is the index.php page that was spidered and cached so quickly. It has not been spidered since though. Maybe it was jus a freak occurrance, who knows. I was just wondering if anyone else had had the same happen.
I can see how that would work though, lots of news feeds, and links built in. They change often so the links between the feeds must be important.
Posted 05 October 2004 - 09:26 PM
From what I am seeing the links on the page with hosted feeds don't seem to get any advantage other than being noticed quicker. They still take a few days to show up on the search engines, and don't seem to get any ranking advantage from it.
I think RSS is a great tool, but not the SEO quick fix claimed by some people who sell RSS parsing modules. I do think that the search engines are going to start realizing the feeds are all the same one of these days if not already.
We will keep paying attention to it, and learn better ways to use the feeds. It's these exchanges of information that help us figure it out.
0 user(s) are reading this topic
0 members, 0 guests, 0 anonymous users