Are you a Google Analytics enthusiast?
More SEO Content
Posted 18 September 2003 - 06:32 PM
Posted 22 September 2003 - 03:55 AM
Hate to rain on anyone's parade here - but to get this absolutely clear:
There are two classes of INK spider - the natural class and the PFI class.
The natural class will crawl from links it has found naturally on the web. If it finds a sitemap through it being linked to by other pages it has in its' crawl database - it will crawl the links from the sitemap.
The PFI class will crawl the pages it has been paid to crawl. It crawls no other pages - ever! It will not follow any links - ever!
Paying for a sitemap to be in PFI with the vain hope that it is a cheap way of getting all your pages indexed is a waste of both time and money (IMHO)
Posted 22 September 2003 - 08:27 AM
Posted 22 September 2003 - 08:37 AM
"We submitted a site map thru PFI from a site that was brand new, had no incoming links and no pages listed in the Inktomi index and you know what - it found and included all the pages listed on that site map. "
Maybe what your saying (MakeMeTop) about the PFI spider not crawling additional links is true but in this case it did.
Bottom line for one is to check!
If a client wants all their pages included via PFI and they are not already in there, then pay for them to be included. If they are, save them their money.
If they do not have the $$ to shell out for PFI and no pages are in there, then try submitting the home page or a site map and even after this, check to see if other pages were included.
It's easy to say that a PFI spider will not crawl additional links which may be true, but some spider crawled those links from the site map we submitted - whether the PFI or INK, I do not know but it occurred.
BTW I have seen this happen other times as well.
Posted 22 September 2003 - 03:42 PM
How long will it take before I see some evidence that there has been uptake on the PFI ala a current homepage cached in allthe web and a few more linked pages in MSN? It has been a week since submission. I have a PR4 or better in Google and rank highly for almost all my key phrases. The site has no subterfuge and is updated weekly.
Thanks for this great resource Jill and to all the contributors who are so free with their sharing.
Posted 22 September 2003 - 03:48 PM
Fast and Inktomi indexes PFI pages in 24-72 hours and then it depends on how frequently All The Web updates the public index which they are pretty fast at doing.
How long will it take before I see some evidence that there has been uptake on the PFI ala a current homepage cached in allthe web and a few more linked pages in MSN?
I would say anywhere from 3 days to no more than a month for both ATW and MSN.
Posted 22 September 2003 - 06:29 PM
I guess I'll just have to try it for myself and report back...
Posted 22 September 2003 - 06:35 PM
Just to clarify, I said it "can" work, not that it always does. It has worked for me a few times.
So, in sum, Barry says that the Site map PFI strategy doesn't work and David says that it most certainly does.
Posted 23 September 2003 - 01:00 AM
Either it does or not. I can't see this as being variable. IMHO, it is more likely a side effect, e.g. many high profile SE don't havew a robots.txt file (e.g. http://www.ninemsn.com.au/robots.txt is non existent). Therefore, every single SERP is allowed to be indexed. There are many other SERPs in various engines that are crawlable. So, if one of your pages were thus indexed, this may work, but surely there are better ways to achive the same result.
Just to clarify, I said it "can" work, not that it always does. It has worked for me a few times
I am with Barry. There is no way this will work. It may by-proxy, but not directly.
Edited by projectphp, 23 September 2003 - 01:08 AM.
Posted 23 September 2003 - 03:56 AM
Posted 23 September 2003 - 07:07 AM
Now if I PFI the homepage, which version of the home page will show up in Inktomi's index.
The PFI version. However, I would be a little cautious about putting in your homepage if it is already in the index. It can have somewhat bizarre effects sometimes.
That is not to say I have never done this - and it can work well - but think about it before hand - and if your site homepage is in L$ US - forget it. The L$ listing will override your PFI entry.
Posted 25 September 2003 - 01:33 AM
It has been a week did my PFI for Inktomi and Fast. Both AlltheWeb and MSN have updated with all the old out of date (2002) pages removed.
Each SE has my current homepage and the site map only. Is that as good as it is going to get? No answer required as I will update any changes.
Posted 01 October 2003 - 11:58 AM
PFI Homepage only to fast and Inktomi 3 weeks ago.
My Homepage and map updated a week ago on both MSN and AlltheWeb.
No other pages crawled and MSN retains a few old links
Reason for doing this as as both spiders stopped updating circa June 2002.
I have been getting a little paranoid this last week as I realized that I put a free Google search box on my site about June 2002. Are these competing spiders/SEs penalizing you for loyalties elsewhere? Seems that the ethics of some SEs are less than exemplary.
Posted 01 October 2003 - 12:02 PM
Are these competing spiders/SEs penalizing you for loyalties elsewhere?
No way. Gotta be just a coincidence. The other engines haven't sunk that low...yet.