8 replies to this topic

[idrive]

I cannot figure out why the spiders are not able to spider the deepest, most important pages of two web sites. One uses asp and the other cold fusion. Neither require session IDs or cookies.

Google is able to index this url: www.somesite.ca/html/media/profiles_master.cfm?ptr=UCV

but apparently not this one:
www.somesite.ca/html/packages/packages_master.cfm?pkg=AGH.1.03

There are several pages similar to the latter that need to be indexed.

Another site which uses asp:
Indexed: www.anothersite.ca/forms/index.asp?dsp=template

Not indexed:
www.anothersite.ca/forms/index.asp?dsp=template&act=view3&template_id=161&lang=e

The second url from the second site does appear to be much more convoluted - but I am not able to tell if there are too many parameters, etc. for it to be spidered.

Input appreciated!

[SearchRank]

Hi idrive,

How are those pages linked? Graphical links or text links? If graphical, then you may want to add text links as the Google Webmaster Guidelines specifically state:

Make a site with a clear hierarchy and text links. Every page should be reachable from at least one static text link.

Also, where do those links preside? On home page? On page that is linked to home page? If your links are hidden too deep in the web site, it may be that Google has simply not found them.

Answers to these questions should help in a response of help.

[idrive]

How are those pages linked? Graphical links or text links? If graphical, then you may want to add text links as the Google Webmaster Guidelines specifically state:

Make a site with a clear hierarchy and text links. Every page should be reachable from at least one static text link.

Hmmm...I have absolutely identified linking as a potential problem. The search engine has indexed www.site.ca/html/packages/packages_intro.cfm and from there we have now made text links into the categories of pages whereas before one had to select from a drop down menu in a form.

From the package intro page listed above, the next page is package streams or categories. This url is available within two clicks from the home page - from the home page you can click on a packages link which takes you to the packages intro url listed above and from there one can click on the streams of packages or categories.

None of these links are "text". They are images. But Google was able to get to some pages even though the links to them are images.

Part of the larger plan is get to related sites to deep link - link into the precise package they are associated with as opposed to linking to just the main page.

If this is all muddy I'd be happy to pm the actual url to anyone interested. Thanks!

[SearchRank]

...from there we have now made text links into the categories of pages whereas before one had to select from a drop down menu in a form.

There you go! Spiders can't fill out forms or follow links from drop downs (probably JavaScript - right). Having text links will help Google and other spiders find those pages.

Google can and will follow graphical links as well as text. Therefore if you are experiencing problems with spiders indexing pages and all you have is graphical links, try adding some text as well. This can most effectively be done with a Site Map. If your pages are getting indexed with graphic links though then there is not need to worry about it. Google will follow both but seems to prefer text links.

[SearchRank]

I checked out both www.journeysofdiscovery.ca and www.cpac.ca after you sent me PM and can offer the following advice.

For first site I would suggest placing text links to all the main sections at the bottom of the page - links to at least all the areas that are currently accessible from the graphic links on the home page and then I'd build a site map that contains text links to all the major areas of the site and make sure that site map is accessible from the home page.

On the second, I foresee a problem with your JavaScript menus as spiders cannot follow links in JavaScript menus. Again a site map will help you here. I noticed that once you select language, there is a site map link at bottom of page. However, it doesn't seem that Google has found it yet. Is the site new or has it been recently redesigned? Has the site map been there that long? It seems that if it has, Google would have indexed it by now.

BTW - nice sites!

[idrive]

Thanks Searchrank! We'll wait to see what happens ;-) Now I think I will return to a previous post where people were talking about :robot watching software" to find an easy way to tell if googlebot has visited any of my desired pages ;-)

[idrive]

I have not been able to resolve the problem of specific pages being indexed by Google even though there have been some fantastic ideas posted here.

I took my problem to I-Search and have received so far one interesting theory. Is it possible that Google cannot or chooses not to index a site where there is a period in the url? Like this:

www.journeysofdiscovery.ca/html/packages/packages_master.cfm?pkg=HPK.2.03


If the periods are posing a problem in the package IDs, this will be easy to change in the database and therefore the site ;-)

[Scottie]

Not indexed:
www.anothersite.ca/forms/index.asp?dsp=template&act=view3&template_id=161&lang=e

There are too many parameters in this URL- Google will not spider it.

[Ron Carnell]

LOL. I think I suggested the resurfacing of Google's old period bug in your other thread?