Lets clear up a definition to make life easier; using server side scripting - perl, ASP, JSP, php, CF whatever doesn't neccessarily imply a truly "dynamic" site.
If you have an .asp etc page, where you are passing harvested details from a form around in variables from page to page, and then submit the collected information to yourself in an email; that uses server side technology but isn't really dynamic at all. Therefore, "feedback.asp", "comments.php" etc are very simple pages which never really change, so are easily indexed by a search engine.
The next level up from this is a database driven site, where one 'template' page is used to pull in product details, using a querystring to populate the information on the page from a database.
Examples would be;
"product.asp?id=214",
"section.php?category=books&subcat=fiction&author=cslewis"
Once again, these present very few problems for most crawlers, who can index these "constant" pages, complete with querystring as in the second example, cslewis is always an author of fiction books (and will never be a 'director' of 'DVDs' in the category 'films')
Where search engines fail to index sites successfully, is where the unique visitor or session id is encoded into the URL. Notice the word unique. A 32 bit hex number in the url somewhere means that the exact same URL is (theoretically....) NEVER going to appear again. So if a SE tries to index your site and you encode the URL in there straight away as soon as they hit the index page.... trouble. A search engine spider is just another "user agent" or browser as far as your dynamic site is concerned, and the URL it retrieves might have the format:
http://63b2e9f26cef7...co.uk/index.phpevery subsequent page would be indexed with the base URL
http://63b2e9f26cef7...blahblah.co.uk/which, being theoretically unique means that either, every time the search engine spiders the site, it adds a new instance of the entire site to it's database as a seperate entity form the previous one - hence you risk spamming their index with an infinite number of versions of your site. More commonly "this session has expired" message appears when a user clicks the link, which means the URL no longer exists - so even if you are spidered and indexed, the link ceases to be valid and clickable after 30 minutes because the session has timed out - no one is ever going to find that page even if a search engine does link to it.
The partial solution is to not have session id's in your URL needlessly. If you gotta have it there, save it for when you REALLY need it - like when a punter adds soemething to their cart. I don't know of many spiders who actually go shopping !!
And mopacfan is right - sitemaps rule, especially dynamically generated ones !! Trawl your database with code - have a site map that includes
"<a href=news.asp?article_id=1 (to n)>" & Response.Write (fldArticleTitle) & "</a><br>" etc
That way all your dynamic stuff is in the sitemap and will be indexed. Sitemaps are great for spiders, but they are actually great for you users too !!!!!
Rez
