  |
Search Engines Indexing Dynamic SitesJanuary 3, 2007 Hi Jill,
I don’t understand how search engines list dynamic sites. For example, this site has a whole bunch of products. Every single product page calls up the same url: www.example.com/products.php.
Will every single page be crawled and eventually listed?
Each individual page has a unique <title> tag.
Thanks for all your help over the years, Jill.
Ros
++Jill’s Response++
Hi Ros,
If every product page is the same URL, then no, that’s not a crawler-friendly design, and the engines won’t be able to crawl and index the site correctly. However, if there are additional query strings appended to the end of that URL, e.g., www.example.com/products.php?product=blackhat, then there shouldn’t be any problem with those URLs being indexed by the search engines.
Dynamic sites can definitely be crawled and indexed if created with the search engines in mind. Make sure that every page has a unique URL and that every page has a way to click to it via the navigation of your site rather than a search box or pulldown menus.
Those are the basic things you need to think about, but there are additional ones. You may want to visit the Dynamically Generated Site Issues section of the forum for more info.
Best,
Jill Post Comment  Anonymous said: Does this mean, that we have to create text links for all those dynamically generated pages so that search engines can follow them. Not even sure, how many pages. Because, each page is generated based on parameters passed. Jill said: Anonymous, yes. Well, they don't have to be text links, but you need to have links somewhere on a crawlable page that the search engines can find and crawl them. They can't do searches at your site or submit forms or anything. It could be in the form of a sitemap, but if the only place those links are found is on a sitemap, they may not be given enough weight to actually get ranked for much. Add Your Comments |
 |
|
Comment @ 01/04/07 at 4:09 am
Jill’s answer is certainly spot on. I recently created an online store in which each item had a URL of www.domain.com/store.asp?item= and also ensured the title and meta tags were distinct for each item. Within a very short time, some of the products were ranked on the first page on Yahoo and it didn’t take too much longer before referrals were coming in from Google. The dynamic nature of the web site isn’t a problem at all provided the URLs are kept relatively short and you ensure you don’t use id=xx in your URLs. It’s definitely a good thing to have distinct title and meta tags for each item too.
Ian