Jump to content

  • Log in with Facebook Log in with Twitter Log In with Google      Sign In   
  • Create Account

Subscribe to HRA Now!

 



Are you a Google Analytics enthusiast?

Share and download Custom Google Analytics Reports, dashboards and advanced segments--for FREE! 

 



 

 www.CustomReportSharing.com 

From the folks who brought you High Rankings!



Photo
- - - - -

Does Sitemaps And Text Version Of Site Help?


This topic has been archived. This means that you cannot reply to this topic.
19 replies to this topic

#1 alnany

alnany

    HR 2

  • Active Members
  • PipPip
  • 14 posts

Posted 18 August 2003 - 11:06 PM

I've been basically trying whatever I read and made sense. I've heard that building a sitemap helps the spider crawl the site better? How about a text version of the website? Thanks.

#2 Mel

Mel

    HR 5

  • Active Members
  • PipPipPipPipPip
  • 353 posts

Posted 18 August 2003 - 11:28 PM

Hi Al

A sitemap is IMO almost a necessity on any website, though strictly it is not necessary if you have good links to all the pages in your site in a place the the spiders can find easily.

I see no reason to create a text version of any website.

#3 alnany

alnany

    HR 2

  • Active Members
  • PipPip
  • 14 posts

Posted 18 August 2003 - 11:35 PM

Aw man... We've spent quite sometime on the text version... But thanks, at least now we know :T

#4 Jill

Jill

    Recovering SEO

  • Admin
  • 32,775 posts

Posted 19 August 2003 - 11:29 AM

Welcome, alnany! :D

It depends.

You might need a text version if your site is in all Flash. Is it? Or all graphics?

As to a site map, it's important if you feel your users need one to better navigate your site, and/or if your current navigation is not spider-friendly.

Jill

#5 qwerty

qwerty

    HR 10

  • Moderator
  • 8,549 posts

Posted 19 August 2003 - 11:44 AM

I'm not sure what you plan to use the text version for. Is this for making it easier to print pages? If so, that's fine, but it can also be accomplished by calling a different style sheet.

Also, if you're going to have extra versions of every page, I'd recommend using your robots.txt file to disallow spiders from accessing the text version. You don't want the search engines finding duplicate content.

#6 Kenny

Kenny

    HR 1

  • Members
  • Pip
  • 2 posts

Posted 21 August 2003 - 05:59 PM

Hello, can you please further explain how to use a "spider.txt"? ie. where do I place it, what the contents should be, how it works, etc. Thanks in advance!

#7 qwerty

qwerty

    HR 10

  • Moderator
  • 8,549 posts

Posted 21 August 2003 - 06:13 PM

You must mean the robots.txt.

It's a plain text file that gets uploaded to your site's root directory, e.g., www.mydomain.com/robots.txt

What it does is specify which robots are to be refused access to which directories. Have a look at this page and let me know if you have any other questions.

#8 Mel

Mel

    HR 5

  • Active Members
  • PipPipPipPipPip
  • 353 posts

Posted 22 August 2003 - 04:38 AM

If you want a good list of "bad" spiders to exclude in your robots.txt file Brette Tabke allows free use of his searchengineworld robots file, which has a pretty exhaustive list, but remember that you will have to customize the file to suit your site architecture.

#9 balz

balz

    HR 2

  • Active Members
  • PipPip
  • 43 posts

Posted 22 August 2003 - 03:22 PM

Just out of curiosity, what makes a spider "bad"?

b.

#10 qwerty

qwerty

    HR 10

  • Moderator
  • 8,549 posts

Posted 22 August 2003 - 03:29 PM

I can think of two kinds of spiders that could be called bad: sometimes legitimate spiders from legitimate search engines will crawl too much. They'll request a ton of pages and burn up your bandwidth. I recently read about either Scooter (AltaVista) or Slurp (Inktomi) doing that to some directories (it's never happened to me, though). Under such circumstances, one might decide to disallow the spider, even though it would mean that you could get dropped from that SE's index.

The other kind would be something like an email harvester. Of course, it's not likely such a spider would obey the robots protocol, so disallowing it wouldn't accomplish much.

#11 Marc

Marc

    HR 3

  • Active Members
  • PipPipPip
  • 90 posts

Posted 27 August 2003 - 03:14 PM

Site Maps are a must have....

here's my take on that subject, why you should have one! :notworthy:

http://www.highranki.....ite map&st=15

Take Care!

#12 Cruise Planner

Cruise Planner

    HR 2

  • Active Members
  • PipPip
  • 18 posts

Posted 27 August 2003 - 03:59 PM

A related question about sitemaps...

When submitting to search engines, is it better to submit the url for the sitemap or the url for the homepage, or even both?

#13 qwerty

qwerty

    HR 10

  • Moderator
  • 8,549 posts

Posted 27 August 2003 - 04:01 PM

Here's what it says on Google's Add URL page:

Only the top-level page from a host is necessary; you do not need to submit each individual page. Our crawler, Googlebot, will be able to find the rest. Google updates its index on a regular basis, so updated or outdated link submissions are not necessary. Dead links will 'fade out' of our index on our next crawl when we update our entire index


So submit the home page if you're going to submit anything, and make sure the home page has a text link to the site map on it.

#14 Scottie

Scottie

    Psycho Mom

  • Admin
  • 6,294 posts

Posted 27 August 2003 - 04:04 PM

You will find most of us don't submit to search engines at all... there is no need.

Spiders will find your site by crawling links to it. If you submit and there are no other links to it, it won't stay in the index for long.

If you do submit (it doesn't hurt) you can submit both your homepage and the sitemap. If I were only going to submit one, I'd submit the home page.

:notworthy:

#15 garyhall

garyhall

    HR 2

  • Active Members
  • PipPip
  • 38 posts

Posted 27 August 2003 - 05:14 PM

You will find most of us don't submit to search engines at all... there is no need.

You are correct and not a doubt about it. BUT try to explain that to
a customer who is paying you big bucks. You got to show him
something. Sitting on your hands is not an option when starting out
with a new customer. Basically I do the submitting for free after I
have been paid of course. It is just good PR to show the customer
you are doing something.

Years ago while walking through our large corporation goofing off,
I discovered if I carried some papers with me I wouldn't be bothered
by anyone. Appearances sometimes is most important, if you get the
drift.

Yahoo really irks me paying them the $299.00 bucks, but they are
sorta like the big gorilla, if you know what I mean.

Regards,

Gary




SPAM FREE FORUM!
 
If you are just registering to spam,
don't bother. You will be wasting your
time as your spam will never see the
light of day!