7 replies to this topic

[chrishirst]

googlebot gets the ability to submit forms on some sites.

Crawling through HTML forms

Specifically, when we encounter a <FORM> element on a high-quality site, we might choose to do a small number of queries using the form. For text boxes, our computers automatically choose words from the site that has the form; for select menus, check boxes, and radio buttons on the form, we choose from among the values of the HTML.

bolding is mine

[Randy]

The tiny mention --repeated later in the comments as confirmation-- that I liked seeing was that they'll only attempt GET forms, but leave POST method forms alone. This one little thing should keep the spider from wildly adding stuff to shopping carts, skewing data all over the place since all carts really should be using POST for this type of thing, and should most certainly be using POST method when collecting payment data.

FTR I still wouldn't recommend using <form>s as the ony way to get to a page if you wanted it to be spidered and indexed. The same goes for javascript links, even though they may try to discover the the resulting page.

[Jill]

I don't really think this is a good thing.

[Randy]

Well, it does mean you'll need to be more exacting in how you use forms if you've been using them to stop spiders from reaching certain areas of a site. Let's just say I've put it on my To Do list to review the various forms on my various sites to make sure I'm using the POST method in all of those places that need it.

As a possible positive, I suppose this does mean Google might be able to find Site Search results pages more easily. I'm not sure if that's a good thing or not, since I've always found it quite easy to get those pages I want to be indexed anyway. And being a stickler I like to get them to the content I want them to get to, not what some algo thinks it might want to get to.

Here's a good test for someone to perform, just to see what will happen. Set up a simple one-input form and place a default value in the form box. The watch your logs to see if Googlebot "submits" the form, with your default value intact. My hunch is you'll probably see them looking for that default value, but it's purely a guess.

[Scottie]

I'm not sure I understand the point. I don't think I want form submissions from Googlebot where "our computers automatically choose words from the site that has the form".

What's the goal here?

[Jill]

Exactly! It sounds like what those form spammer bots do.

[don h]

I think they should keep it to sites that opt-in for it, through the WMT interface, like they do with their enhanced image search.

[Otaku]

The next news we'll hear is "Googlebot is now buying stuff from online stores (has its own credit card with the name Mr. Google Bot) and ranges sites according to the shipping time - this additional factor will make search results even more relevant for users"