Are you a Google Analytics enthusiast?
More SEO Content
Posted 21 October 2003 - 09:16 AM
will be retuned as
now thats all well and good and, up to a point, makes some sense. the problem is (you knew it was coming!) that according to my referrer logs googlebot has been scraping urls such as
from my site and displaying it to users in its search results. how can this be?
even more perplexing, is that there are several occaisons when a huge url (with several occurances of &, ? and = in it) has been scraped!!
has anyone else encountered this?
Posted 21 October 2003 - 09:24 AM
Google, at least, seems to have figured out how to deal with it and they are not stop characters at this point.
Not sure what you mean about the urls being scraped, could you elaborate?
Posted 21 October 2003 - 09:29 AM
"scraped" - sorry, force of habit! my old job involved building scripts to retreive information from web pages; we used to call this 'scraping'!
i simply meant that it appears that googlebot can follow links with these supposed 'stop characters' in them.
so google's figured it out, thats good news, what about other engines?
is the issue of stop characters no longer an issue?
Posted 21 October 2003 - 02:15 PM
Ive read from several sources that certain characters (&, =, ?) contained within a url will instruct the Spider to stop in its tracks a return whatever section of the url it has crawled over
Google, Inktomi, Ask Jeeves, and the lot can follow dynamic links, even with characters such as =, &, %, and ?.
It all depends on how well the code is written, not on the characters used. If the code is too convoluted and involves too many parameters, it won't be spidered. But simply having a ? does not mean that a spider stops and runs away.
That excuse is what amatuer programmers use to explain their lack of programming skills.
Obviously, you are getting your dynamic site indexed, so your code must be doing the job. The URL you referred to as being returned as
The URL being re-written like this only happens when a mod-rewrite has been initiated by the server, not by the search engine.
will be retuned as
Edited by Matt B, 21 October 2003 - 02:25 PM.
Posted 21 October 2003 - 08:02 PM
What you are talking about must be a old document or some sites which is selling some products to expose the dynami URLs,
Now google can follow any dynamic URLs even with multiple parameters passed after it, INKTOMI and FAST too can handle any type of query strings, So no need to worry about dynamic generated URLs,
But make sure you optimize the contents in your Database(like the description of a prduct) too it will help you in rankings,
0 user(s) are reading this topic
0 members, 0 guests, 0 anonymous users