Jump to content

  • Log in with Facebook Log in with Twitter Log In with Google      Sign In   
  • Create Account

Subscribe to HRA Now!

 



Are you a Google Analytics enthusiast?

Share and download Custom Google Analytics Reports, dashboards and advanced segments--for FREE! 

 



 

 www.CustomReportSharing.com 

From the folks who brought you High Rankings!



Photo
- - - - -

Random Question: Spiders


This topic has been archived. This means that you cannot reply to this topic.
9 replies to this topic

#1 jbelle

jbelle

    HR 4

  • Active Members
  • PipPipPipPip
  • 206 posts

Posted 24 July 2003 - 02:11 PM

This is a little random but I updated a site a few weeks ago and optimized the text. Naturally, I'm anxious to see the results, so I irrationally check my sites stats daily (sometimes more frequently than that!) to see what spiders have been crawling. There are faithfully a few hits an hour from all the major bots, but nothing more.

I've never watched these spiders so carefully; will I see a big spike when they've finally decided to do a "deep crawl"?

#2 robwatts

robwatts

    HR 5

  • Active Members
  • PipPipPipPipPip
  • 308 posts

Posted 24 July 2003 - 05:08 PM

Hi jbelle

It all depends on the size of the domain really.

if I were you, I wouldnt look at your log files every day (it'll drive you potty)

Get yourself a good log analysis tool { there are loads of free ones such as webaliser, analog and a few more sophisticated ones clicktracks webtrends there are loads more too, but they escape me at the minute} and then have a little look once a week or so and see what they say.

They should all be able to show you the freq of the bot visit and the pages they grabbed too.

I'd use the time you'd otherwise spend pouring over the logs to perhaps, build links or add some more content - far more productive IMO. :)

#3 Matt B

Matt B

    The modem is the message.

  • Active Members
  • PipPipPipPipPipPip
  • 558 posts

Posted 24 July 2003 - 07:59 PM

I agree with Rob, I wouldn't worry too much about spiders.

If you have your initial directory links in order, and good, spiderable content, then the spiders will naturally come around. Especially after making changes - give it some time.

#4 jbelle

jbelle

    HR 4

  • Active Members
  • PipPipPipPip
  • 206 posts

Posted 28 July 2003 - 01:22 PM

Of course there are better things to do with my time, but I'm too busy checking my stats daily still to leave time to evaluate more efficient use of my time! :rolleyes:

I noted some jumps out of several spiders upon deleting some old pages, so I'm still curious to see what happens with spiders in their natural habitat.

I think my work can handle the 2 minutes it takes to check my stats once a day to feed my addiction--but don't worry, I'll deny it if asked!

#5 robwatts

robwatts

    HR 5

  • Active Members
  • PipPipPipPipPip
  • 308 posts

Posted 28 July 2003 - 01:47 PM

Of course there are better things to do with my time, but I'm too busy checking my stats daily still to leave time to evaluate more efficient use of my time! 


ah! you cant beat a bit of convoluted fuzzy logic! :rolleyes:

#6 Scottie

Scottie

    Psycho Mom

  • Admin
  • 6,294 posts

Posted 28 July 2003 - 01:55 PM

I think my work can handle the 2 minutes it takes to check my stats once a day to feed my addiction--but don't worry, I'll deny it if asked!

Denial is a sure sign of addiction... :wacko:

Try not to look at all. "Just a quick look" always turns into a more in-depth time-waster than it should be. Like Ebay or forums. One thing leads to another and suddenly the day is gone. :stunned: The only way to manage is to go cold turkey. (good advice from one addict to another- do as I say... not as I do :rolleyes: )

Does anyone have a favorite free stat analysis program? I'm not thrilled with Webalyzer (offered by my host) and haven't really found a freebie that covered all the bases yet. I haven't researched it much because I try to stay away from my stats... I can spend hours going through them! But every now and then, I do want to check things.

#7 jbelle

jbelle

    HR 4

  • Active Members
  • PipPipPipPip
  • 206 posts

Posted 28 July 2003 - 02:04 PM

Try not to look at all.  "Just a quick look" always turns into a more in-depth time-waster than it should be.  Like Ebay or forums.


LOL; these forums are getting the best of me today!

Well, this was supposed to be a productive post: I just went to check on the free stats package the web firm I used to work with used, and, it's not free anymore! Oh well; just another time waster :rolleyes:

#8 K.S. Katz

K.S. Katz

    HR 4

  • Active Members
  • PipPipPipPip
  • 172 posts

Posted 31 July 2003 - 08:02 AM

Here at Webcast 1, Inc., we use a Perl script modified from AWStats. It works really good for us and is completely free (you just have to do the setup).

To check it out, go to: http://awstats.sourceforge.net/

They do a wonderful graph where they compare AWStats to Webalizer and a few other stats programs. I've seen a few, and by far (in my not so humble opinion) I believe this is the best free stats program out there.

Sincerely,

K.S. Katz :lmao:

#9 Webmaster-Toolkit.com

Webmaster-Toolkit.com

    HR 1

  • Members
  • Pip
  • 3 posts

Posted 31 July 2003 - 08:16 AM

Yeah I'm an avid awstats user too :lmao:

#10 Scottie

Scottie

    Psycho Mom

  • Admin
  • 6,294 posts

Posted 31 July 2003 - 08:20 AM

Thanks!

I have used AWStats before and like it- I didn't realize it was open source.




SPAM FREE FORUM!
 
If you are just registering to spam,
don't bother. You will be wasting your
time as your spam will never see the
light of day!