5 replies to this topic

[lister]

Hi I am using Google Analytics (GA). Am I being very dumb here - how do I tell if a bot - or more specifically if a google bot has decided to pay me a visit. I am sure it has since it has generated a little javascript for me to use -

Thanks!

[Jill]

I don't believe Google analytics provides you with the bot info.

[torka]

IIRC, Google's Webmaster Tools will tell you the last time Googlebot visited your site. (Not GoAn, though, because AFAIK bots generally don't execute JavaScript.)

--Torka

[1dmf]

GOAN will tell you if the GoogleBot has visited.

I have been puzzling this with my thread and some stats which show as 0x0 screen resolution, this could be a bot, Randy and I cannot say for sure though.

Check your screen resolution / browser stats and see if you find the same.

[Randy]

Remember though 1dmf, there are some Googlebots today that execute Javascript. Those are the only ones GoAn is going to see.

For my own sites the GoAn reported (JS enabled) Googlebots hits are a fraction of Googlebot hits I see in my log analyzer stats software.

And that's the key to the original question. Google Analytics doesn't see most spider visits at all. For exactly the reason Torka already said, because GoAn requires Javascript to see a hit or anything about it, and the spiders as a general rule do not execute Javascript. The only exception to this as far as I know is one version of Googlebot that crawls around processing JS to look for nefarious javascript redirects and such.

You'll need a stats program that analyzes your Log Files to see spider hits. Most hosts these days automatically set up such log based stats programs for you these days. So the info is probably already there in your hosting account somewhere.

[1dmf]

hmm, well I guess I won't know then without trawling my server logs and I really do have better things to do!

Plus I could write a scraping tool that pretends to be IE browser, so you will never get a true picture, as many bots with 'disguise' themselves.