2 replies to this topic

[BBCoach]

I'm noticing something that's not suppose to be happening. For the last two months Googlebot has been randomly generating query string elements and posting the form. At least I think their random. There are no links to the results page because everything is dynamically generated from user selections. These are product specific searches that a user selects a model and then say the style. One form in particular they are incrementally posting values like var1=59&var2=59 which generates an error because that value combination doesn't exist and never has. In the last hour that has happened 30 times with different values passed.

Has anyone else seen this? It's not a big deal unless G is trying to learn how to post a form from the values in the select lists.

Interesting???

[Randy]

I've not seen it before BBCoach, however I do know Google will occasionally try something similar if it sees a page where the variables are confusing. When they do this my understanding is that they're basically they're testing to figure out what is the shortest number of variables that may be used to produce a page.

I've never seen this happen via a form submission though. So my hunch is there is a link or two out there somewhere pointing to the page with variables in the URL instead of a form that sends the variable information transparently. This may be on your site, or it could be link from other site. Which makes it hard to identify.

A question: Does your form processing page accept GET or only POST variables?

The reason I ask is that you can fix it (easily) if the page will only accept POST method requests from your form pages. Meaning you can code it so that if it receives a GET request or a request that doesn't have a certain variable or combination of variables that should always be there you can simply have the page send back a proper 404 header response. That'll make sure no spider ever lists a page they shouldn't.

[BBCoach]

Thanks for the insight Randy. I've been observing it for the past two months and waited until it happened again before saying anything. The page accepts GET and POST. The reason is a lot of our customers will do a search for a particular set of accessories for a product and bookmark that page for later purchases. I don't wanna break that convenience and honestly wouldn't mind if the SEs indexed those pages. If they indexed those pages it would give us a different stemming effect in the SEs by grouping a set of product accessories with the parent product and there are literally thousands of such associations.

Gbot's submissions are GET, but there's no way and no how they would know the query string to use. Gbot is definitely incrementing the values in the query string because I've never listed (with a real search) some of the values their hitting. Looking at the log files it's like they'll basically start in a FOR loop and increment each value. Like var1=10&var2=1, var1=10&var2=2 and on and then incrementing var1=11 when it hits 99, generates an error, or no results for var2. Very interesting. And, honestly, there are no links on the site that point to these product specific search result pages because that would be thousands of links and for this site I don't use a sitemap page with those special search results either. Interesting to say the least. Just wanted to see if anyone else has noticed this behavior.