5 replies to this topic

[scialom]

Hello,

I have a website with many items for sale. Near each one there is a link add it to the cart (like any normal shopping cart...) Now spiders follow these links and add thousands entries to my db.

Should I block them doing so? or do I get and advantage from that?

Thanks for your help,
Assaf

[jehochman]

Why? The spider might buy something.

I don't think any bad will happen if the spider clicks it's way into an empty shopping cart. Spiders don't fill out forms. Are you currently experiencing a problem?

[scialom]

this could be great if they will buy some!

No... just a lot of entries in my db...

[Randy]

Change your add to cart link from a straight <a href ... > to a <form> link and the spiders will stop doing that. As Jonathan said above, spiders don't do forms, so it's a quick and easy way to control where they can and cannot go.

[Christopher]

We block the spiders via our robots.text however google has still assigned PR and a medium one at that to our cart page... go figure.

[1fast72nova]

They are probably seeing links pointing there, I've noticed that same thing. On a lighter note, maybe if the spiders are making blank entries you can prefill your fields with the most expensive items you have! Nah..