11 replies to this topic

[Talking Bear]

Hello,

This is my first post to this forum. I have been participating as a spectator since the forum began and I am impressed by the quality of questions asked and the related responses.

My site www.connectionchannel.com has a page rank of 6 for the home page and a page rank of 5 for links near the top of the site. All other pages have a page rankings varing from 4 to 0. What I have noticed is that when google re-indexes my pages, either at the end of the month or during periods within the month, I find that pages with a ranking of 5 or more are routinely re-indexed.

My question is, Why aren't pages with a page rank of 4 or less being re-indexed, and why do certain pages have a page rank of 5 while others are 4 or less ?" Currently our link exchange program points to the home page only.

Thanks

[Jill]

Welcome, Talking Bear! Glad you like the forum.

It would make sense for Google to crawl what it considers the "more important" pages of your site more often, and PageRank is how they determine that.

My question to you would be, why do you care? Are you inner pages which are not getting crawled often, changing a lot? If they're not changing, it's really not important for them to be crawled often.

How are they ranking for their relevant keyword phrases? That's much more important than their PageRank or how often they are crawled.

And even more important than that, are they selling?

Jill

[Mel]

Hi Talking Bear
Another factor in getting your lesser ranked pages spidered more often may be that the IIS5 server that you are hosted on is not giving a last modified date response in the header.

Google advocates setting your servers up so that it gets a correct response to an if-modifed-since request (either a 304 or a page) so that it can spider more efficiently, and seems to reward sites that do this by spidering most of the pages that have changed since the last spidering.

[Talking Bear]

Thanks Mel for the information. I was unaware of the of the if-modifed flag. I will check into it tomorrow.

Jill

As to why I want the pages indexed. I am creating doorway pages for specific keywords. For the pages that are indexed the rankings are great. Now my goal is t increase my keyword coverage.


Jill

I enjoy your news letter and find your advice well founded. Keep up the good work...

[hermit]

Doorway pages!? Oh man, was that the wrong thing to say! You may want to rethink that idea search engines are not fond of doorway pages.

[Talking Bear]

Possibly doorway page is the wrong term, but I have a dynamic site and the product url's include a ? which from my understanding and experience won't be indexed. As an alternative I created optimized static pages that contain a number of thumb nail images that when clicked on transfer to the product pages. These pages allow me to add related keywords surrounding the theme of the page. The pages are indexed and I get good rankings in the search engines.

Is there a workaround for the "?" in the url so the page will get indexed.

[amberbaby]

Mel, how can I find out if my server is giving a last modified date response in the header ?

[Mel]

You can go Here and enter your URL.
You should get back a reply that looks something like this:

Server Response: http://search-engine-optimization.ecommercehosts.com
Status: HTTP/1.1 200 OK  
Date: Thu, 14 Aug 2003 12:49:53 GMT  
Server: Apache/1.3.28 (Unix) mod_auth_passthrough/1.6 mod_log_bytes/1.2 mod_bwlimited/1.0 PHP/4.3.2 FrontPage/5.0.2.2634 mod_ssl/2.8.15 OpenSSL/0.9.6b  
[b]Last-Modified: Sat, 02 Aug 2003 13:47:31 GMT [/b] (emphasis added) 
ETag: "e4005-644a-3f2bc0f3"  
Accept-Ranges: bytes  
Content-Length: 25674  
Connection: close  
Content-Type: text/html  

If you are returning a last modified date it is likely that you are also sending the correct response to the if-modified-since query. Normally most Apache (Linux) servers are in conformance without any mods to your page, but IIS servers often need to have code added to each page to make it work.

[amberbaby]

Talking Bear, there is a lot of talk about dynamic pages not being indexed, however, many of my dynamic pages are indexed in google. So wonder if it's not smarter to just let the engines deal with finding a way to index a dynamic site properly rather than us trying to trick them into pretending we have a static site and in the end getting in trouble for spamming.

As for the doorway pages: I think that doorway pages are still a big part of SEO, even when they are being badmouthed in newsletters and forums. The fact is that many good ranking pages for popular search phrases are doorways or affiliate pages which are basically the same.
In my industry, a guy showed up out of the blue with a very well designed affiliate site and he's now on the first page on google for most of the relevant key words. His pages are nothing but doorway pages linking to a "real" sites that sells products.

[Talking Bear]

I would agree with your assesment of doorway pages. The pages I create are specific to a theme and I precieve them as a benefit to a searcher. The pages are targeted and recieve good keyword rankings.

As to waiting for the search engines finding my product urls - I've been waiting 2 years.

Whats a business man to do....

[Jill]

Talking Bear, there's a good chance that Google realizes those pages are doorway pages, and that's why they are not placing much emphasis on them.

Dynamically-generated pages can indeed by spidered and indexed if you take the time to ensure it. Since yours haven't been, there are probably barriers up on your site to prevent them from being spidered and indexed.

If you can remove the barriers and optimize your dynamic content and the resulting html pages, you will generally find that they will do much better in the search engines than any doorway pages that you create.

If you haven't read it already, you might want to read the interview I had with Alan Perkins, optimizing dynamic content.

Jill

[Talking Bear]

Jill

Thanks for the reference link. After reading the review I find that I have addressed the problem as Allen suggested.

2. Link to dynamic URLs from static URL content.

With limited resources, it may prove difficult or impossible for you
to implement a solution based on static URLs. Don't worry! There are
other things you can do.

Over the years, the engines have tried to find ways of crawling
dynamic content while avoiding dynamic spider traps. One technique
they use is crawling dynamic URLs that are linked to from pages with
static URLs. For example, if you give your site map page a static URL,
but have links to dynamic URLs within its content, there's a good
chance that the leading engines will crawl those links. If they like
the content they find there, they will index that content. The search
engines' reasoning here seems to be, "If you're prepared to link to
this content, then so are we."

I've created a site map which includes static links to my product dynamic links and I have created what I call Occasion Pages.

Google visited my site last night since I started this thread. What I found was that my Page Rank has increased to 5 for all the occasion pages. Some weren't indexed, but thanks to Mel I found that the last-modified flag isn't being set so I'm following up with my ISP.

Thanks for the great information