Skip navigation
newsletter signup

Search Engines Indexing Dynamic Sites

January 3, 2007

Hi Jill,


I donít understand how search engines list dynamic sites.  For example, this site has a whole bunch of products. Every single product page calls up the same url:


Will every single page be crawled and eventually listed?


Each individual page has a unique <title> tag.


Thanks for all your help over the years, Jill.




++Jillís Response++


Hi Ros,


If every product page is the same URL, then no, thatís not a crawler-friendly design, and the engines wonít be able to crawl and index the site correctly.  However, if there are additional query strings appended to the end of that URL, e.g.,, then there shouldnít be any problem with those URLs being indexed by the search engines.


Dynamic sites can definitely be crawled and indexed if created with the search engines in mind. Make sure that every page has a unique URL and that every page has a way to click to it via the navigation of your site rather than a search box or pulldown menus.


Those are the basic things you need to think about, but there are additional ones.  You may want to visit the Dynamically Generated Site Issues section of the forum for more info.





Post Comment

 Ian Feavearyear said:
Comment @ 01/04/07 at 4:09 am

Jillís answer is certainly spot on. I recently created an online store in which each item had a URL of and also ensured the title and meta tags were distinct for each item. Within a very short time, some of the products were ranked on the first page on Yahoo and it didnít take too much longer before referrals were coming in from Google. The dynamic nature of the web site isnít a problem at all provided the URLs are kept relatively short and you ensure you donít use id=xx in your URLs. Itís definitely a good thing to have distinct title and meta tags for each item too.

 Anonymous said:
Does this mean, that we have to create text links for all those dynamically generated pages so that search engines can follow them. Not even sure, how many pages. Because, each page is generated based on parameters passed.
 Jill said:
Anonymous, yes. Well, they don't have to be text links, but you need to have links somewhere on a crawlable page that the search engines can find and crawl them. They can't do searches at your site or submit forms or anything.

It could be in the form of a sitemap, but if the only place those links are found is on a sitemap, they may not be given enough weight to actually get ranked for much.