Hi
We have a situation currently where Googlebot is suddenly doing an amazing amount of
activity on one of our client sites, 30-40 M per day which it wasn't before. I am at a
loss to understand why this is occuring - any ideas?
The only idea I have to stop it is to add a robots.txt to ban GoogleBot from
the site. But that's not ideal. The problem is though that this huge increase in bandwidth is costing us money to host.
Thanks in advance!
Subscribe to HRA Now!
Are you a Google Analytics enthusiast?
Share and download Custom Google Analytics Reports, dashboards and advanced segments--for FREE!
www.CustomReportSharing.com
From the folks who brought you High Rankings!
More SEO Content
International SEM | Social Media | Search Friendly Design | SEO | Paid Search / PPC | Seminars | Forum Threads | Q&A | Copywriting | Keyword Research | Web Analytics / Conversions | Blogging | Dynamic Sites | Linking | SEO Services | Site Architecture | Search Engine Spam | Wrap-ups | Business Issues | HRA Questions | Online Courses
Sponsored Content
Google Suddenly Spidering...a Lot
Started by
Kate
, Jan 28 2005 09:28 AM
7 replies to this topic
#2
OldWelshGuy
-
- Moderator
-
- 4,713 posts
Work is Fun
- Location:Neath, South Wales, UK
Posted 28 January 2005 - 10:08 AM
Hi Kate, you could try the robots revisit after xxx days meta. This would tell googlebot not to visit until that number of days has elapsed. My guess is that G has found a few more links to your client site and feels it is now worthy of attention. Does your client site use session ID's? as if it does, google could be finding lots of new pages at every visit if it is picking up the sess id in the URL.
#3
Alan Perkins
-
- Admin
-
- 1,559 posts
Token male admin
- Location:UK
Posted 28 January 2005 - 10:36 AM
The revisit-after tag does not work for Google.
Send an e-mail to googlebot@google.com describing your problem in detail. Include a snippet from your log files showing the excess activity.
Send an e-mail to googlebot@google.com describing your problem in detail. Include a snippet from your log files showing the excess activity.
#5
tempy
-
- Active Members
-
- 344 posts
HR 5
Posted 28 January 2005 - 11:32 AM
Just wait until the MSN bot starts taking a liking to your site. I read some reports that it's been crashing servers in some instances.
#6
OldWelshGuy
-
- Moderator
-
- 4,713 posts
Work is Fun
- Location:Neath, South Wales, UK
Posted 28 January 2005 - 11:36 AM
QUOTE(Alan Perkins @ Jan 28 2005, 11:36 AM)
The revisit-after tag does not work for Google.
Send an e-mail to googlebot@google.com describing your problem in detail. Include a snippet from your log files showing the excess activity.
Send an e-mail to googlebot@google.com describing your problem in detail. Include a snippet from your log files showing the excess activity.
Cheers Alan
I wasn't sure about that.
#7
Alan Perkins
-
- Admin
-
- 1,559 posts
Token male admin
- Location:UK
Posted 28 January 2005 - 11:54 AM
No problem.
#8
lyn
-
- Active Members
-
- 940 posts
HR 6
- Location:London, Ontario
Posted 28 January 2005 - 12:46 PM
QUOTE(tempy @ Jan 28 2005, 12:32 PM)
Just wait until the MSN bot starts taking a liking to your site. I read some reports that it's been crashing servers in some instances.
Won't be a problem for me - MSN has always hated my site.
Lucky that Google & Yahoo think I'm OK!
L.
0 user(s) are reading this topic
0 members, 0 guests, 0 anonymous users
