11 replies to this topic

[mini]

Hello

I have gone through the pinned thread which refers to the Jill-Alan interview.

But still got to learn more about dynamic content and urls. I have some confusion and would like to present my problems one by one.

My question is : Are the following urls static?

1) abc.example.com/music/

2) abc.example.com/listjavagame.php

Edited by Randy, 01 August 2008 - 06:43 AM.
Edited to break example urls.

[chrishirst]

My question is : Are the following urls static?

They might be, but then on the other hand they might not be

they are static looking is all you can say about them.

[mini]

Thanks for the quick reply....

Each of the above two pages have links to pages with dynamic url n dynamic content. And these links also come dynamically on the two given pages.

Lets take abc.example.com/music/ :
It has many albums (in the form of dynamically generated links). The Google cache of this page shows different links as the database is regularly updated.

Thats is,
abc.example.com/music/ .............Has dynamic content (Links) and IS INDEXED by Google.

abc.example.com/music/albumtrack.php?artist=139&album=10341 (One of the links)............This page is NOT INDEXED yet.

According to Alan, it is not a problem for SEs to index pages with dynamic url n dynamic content if they are linked to by a page with static url. The problem is I can't find out if abc.example.com/music/ is a static url or not.

Edited by Randy, 01 August 2008 - 06:44 AM.
Edited to break example urls.

[chrishirst]

Ok I'll ask you a question (or two).

www.gatewayseducation.org/ dynamic site or static site?

creeme.inyoursite.co.uk/ dynamic site or static site?

www.candsdesign.co.uk/ dynamic site or static site?

ok it was three

[Jill]

As Chris said, they are "static-looking" which means for all intents and purposes, the search engines, they are static.

[1dmf]

It is impossible to tell if a page is static or not just by the URL.

I would, if looking conventionally at extensions, say a php site is dynamic, otherwise why have a PHP extension? the same could be said for ASP. if it's static then surely HTML would suffice.

But having said that , many may simply use the PHP or ASP extensions to enable some form of SSI (Server Side Include) , to enable dynamic insersion of headers / footers / navigation etc..

But again on the other hand I tell the IIS server to run .html extensions through the SSI interpreter, so my pages 'appear' to be static, but are really dynamic!

Yet I have other pages on another site that is written in ASP, but most pages are actually static! (the original designer, wrote them as .asp, so I stuck with it!)

Also having query string vars on a URL doesn't make it dynamic or static , consider using simple HTML and the 'GET' method on a form, that converts the submitted form data and reloads the page appending it as query string data, which you could in turn then process using JavaScript, all this would be an a standard static type HTML page and no SSI interpreter running!

abc.example.com/music/

as for whether that is a static URL or not, well , that URL is meerly a URL to a folder not a webpage, and is impossible to tell what might be on the other end, CGI script? , index.html? and ASP , PHP , coldfusion or anything else, that again is decided by the server configuration and whether you have set that location to run any files by 'deafult' and also whether you have allowed directory listing!

remember a web page www.mydomain.com/mypath/mypage.ext is actually a URI (Uniform Resource Identifier) , it identifies specifically a resource, URL (Uniform Resource Locator) is more the mechanism used to find a URI, URI has replace URL, as it would seem URL is used as technical way of explaining what a URI is, if I read things right that is - lol.

(I beleive, thinking about it , is why many have tried to patent a hyper link or the URL , even B.T. I believe, as they claim to have invented the URL method to reach a URI!)

No attempt to patent URL ever succeded though!

Edited by 1dmf, 01 August 2008 - 09:49 AM.

[Randy]

Looking to the url structure for the types of urls you've mentioned to figure out why they're not being indexed is a fruitless exercise.

The reason lies elsewhere.

Those url strings are 100% able to be indexed. As evidenced by the millions of 2 variable urls that are already indexed by the various engines.

[mini]

Your replies have helped a lot to understand things. So, that one is actually

abc.example.com/music/index.php

Those url strings are 100% able to be indexed. As evidenced by the millions of 2 variable urls that are already indexed by the various engines.

As far as I know, search engines index pages through links. The page abc.example.com/music/index.php show many links to music albums and these links are dynamically generated (album name is pulled from the database). One example is

abc.example.com/music/albumtrack.php?artist=139&album=10341

So, can I say that the url abc.example.com/music/index.php is static and its content is dynamic?

The page shows links to albums listened recently by users. So the set of album links present on abc.example.com/music/index.php change many times a day. SEs usually do not index pages so fast. At one time, while indexing the page, it will follow those links. The next time it indexes the page, it follows another set of links. But what about those links which come and go in between this time? I hope I am understandable.

The pages with individual albums (that is, abc.example.com/music/albumtrack.php?artist=139&album=10341) are not linked to from any other page. Do I need to link to these pages with individual albums from some other static page?

[chrishirst]

As far as I know, search engines index pages through links.

Correct, sort of. Search Engines find pages to index through links.

The page abc.example.com/music/index.php show many links to music albums and these links are dynamically generated (album name is pulled from the database). One example is

abc.example.com/music/albumtrack.php?artist=139&album=10341

So, can I say that the url abc.example.com/music/index.php is static and its content is dynamic?

No
A true dynamic page is one that has content pulled from a database. All you can deduce from those URLs is that the first one appears to be static while the second appears to be dynamic.
querystring parameters can be placed in a URI for many other reasons than pulling content from a database.

The page shows links to albums listened recently by users. So the set of album links present on abc.example.com/music/index.php change many times a day. SEs usually do not index pages so fast. At one time, while indexing the page, it will follow those links. The next time it indexes the page, it follows another set of links. But what about those links which come and go in between this time? I hope I am understandable.

Having user generated links indexed can be a double edged sword.
Yes, it can get "deeper" content indexed.
Yes, it can bring popular items into focus.
But it can also generate duplicate pages if your search system isn't designed to avoid it.
The links that comes and go? Don't worry about them.

The pages with individual albums (that is, abc.example.com/music/albumtrack.php?artist=139&album=10341) are not linked to from any other page. Do I need to link to these pages with individual albums from some other static page?

Bolding added.

So!
You ask these seemingly random question about "what pages are dynamic etc", and ask "Why are some pages not indexed" THEN you throw in a statement like the one above ?????

IF there are NO links AT ALL to these pages from pages with static looking URIs or from pages with dynamic URIs THEY WILL NEVER be indexed

I would suggest that you have completely misunderstood what Alan is saying. The bottom line is:
IF the only way to get to a page with a "dynamic" URI (child page) is from a page that also has a "dynamic" URI, the SE scheduler/indexer may not send the crawler to get the child page.
Adding the link to the child page to a page with a static URI will avoid this "second generation dynamic" problem.

It does NOT mean that you shouldn't link to "dynamic" pages from other "dynamic" pages. It simply means that you should make sure that there are links TO the child page from pages that appear to be static. The keyword here being appear of course.

[mini]

IF there are NO links AT ALL to these pages from pages with static looking URIs or from pages with dynamic URIs THEY WILL NEVER be indexed

I meant that only the page abc.example.com/music/index.php links to all these albums. No other page links to them. It has been difficult to put all this in words...apologies for that.

[chrishirst]

If you don't consider these pages important and only link to them from ONE other page, why should the search engines consider them important enough to index?

[Randy]

You have found the reason the search engines do not have these artist/album pages indexed. It's because the message you're sending the search engines is you don't consider those pages to be very important.

There is only one page on the entire Internet that links to those artist/album pages, and even it only links to each of them sometimes. Given those two facts the only possible conclusion the engines can reach is that you don't consider those pages to be all that important in the grand scheme of things.

You need to figure out some way to make sure there are links to all of those pages all the time if you want them to stand a chance of being indexed. Hopefully once you get the ball started rolling others will take your queue and start linking to some of these deeper pages all by themselves.

If it were me, I'd set up some sort of system where users (and robots) can get to pages that lists and links out to a page dedicated each artist. Possibly broken down by music genre. Then these Artist pages list and link out to your Artist -> Album pages.

Sort of an Artist Sitemap type of thing. Those would be useful for users and spiders alike, and should accomplish the goal of getting the artist/album pages indexed over time.