Sitemaps - How Important Are They?, which style sitemaps to use Options

[Bob Collett]

I have understood that sitemaps are very important to make sure that G and the others can easily find all of the pages on the website and be more likely to do a complete index. I used to do a manual sitemap, listing each page with a brief but custom descripton plus a link to the page. More recently I started using a sitemap generator such as the one found at G, because I thought they would prefer it. But the G and Y supported sitemap format does not appear to include an actual link to the page. I am interested in your thoughts about the current status and value of sitemaps, and the recommended formats, as well as sitemap generators that you like.
Thanks in advance.
Bob C

[lenwood]

There are several sitemap generation tools that are easy to use. Depending on the client and their needs/abilities I use either Google's python script to generate the sitemap, or the php script available from www.xml-sitemaps.com. Both generate a standards compliant sitemap, and both can be set to run automatically with a chron job. You asked about URLs, if you use either of these tools you're set. There are plenty of other options, if you select another you'll want to make sure that it meets the standards outlined at www.sitemaps.org.

While you're adding the sitemap you may also want to place a robots.txt file on the server that points to it.

HTH,
Chris

[Jill]

I have understood that sitemaps are very important to make sure that G and the others can easily find all of the pages on the website

I really think that's a myth. Unless you have an ecommerce site with hundreds of thousands of pages. And even then, if it's designed correctly, a sitemap (XML type submitted to Google) won't get you any benefits.

[KingPin]

I use a .php site map with the links on for my ecommerce sites and I also use RORFiles.

Saying that I don't have any conclusive evidence that it helps or hinders.

KP

[Mary McKnight]

I think sitemaps are not going to get you any great benefits, however I do use both an auto generated on page sitemap for flat architecture and an XML sitemap which boosts priority of specific types of pages. For example, with real estate sites- the listing pages should have greater priority than an about me or blog post page, so we are able to denote greater priority for those types of pages. In those cases there are benefits, but overall, a sitemap is not essential.

[lenwood]

This raises a question for me. I have each of my clients set up a robots.txt and sitemap to ensure that all of their content and my work gets discovered by the search engines. Am I helping myself by doing this? Ultimately, is there any way to influence what - and how often - the search engines index?

Thanks,
Chris

[dom-casino]

I really think that's a myth. Unless you have an ecommerce site with hundreds of thousands of pages. And even then, if it's designed correctly, a sitemap (XML type submitted to Google) won't get you any benefits.

I don't think so, sitemap always helps, whehter it's small sites or ecommerce websites.

[Randy]

xml sitemaps are only helpful if you've put some technical barriers in the way of the spider that is keeping them from getting to all of the pages of a given site. They don't hurt you if you have one and already have a spiderable site, but there's no magic elixir there.

I always have and probably always will create human readable site maps because visitors actually use them. But even with those I don't bother trying to link to every single page in a site. I just hit the high points.

I've never employed an xml sitemap because I've never needed to since I build my sites to be spider friendly. And haven't suffered a bit for it.

[Catz]

I always have and probably always will create human readable site maps because visitors actually use them. But even with those I don't bother trying to link to every single page in a site. I just hit the high points.

I'm with you. This is something that has seemed to fall by the wayside with so many "developers" out there these days.

It's like user friendly web development has been replaced with automated web design...which has brought about a lot of lazy "web designers/developers".

Computer programs do not have a common sense mode built in so sometimes you have to step back and think things through rather than just counting on the computer to do all the thinking/work for you.

I've never employed an xml sitemap because I've never needed to since I build my sites to be spider friendly. And haven't suffered a bit for it.

This is how you can pick the "old timers" out of the crowd. Those with the most experience.

We think through the process and create user friendly web sites that are in turn, spider friendly.

Creating a site map that is useful to your human visitors used to be considered an important part of the web design process, now you see automated lists of links, certainly not created with the human visitor in mind.
I've said it before and I'll say it again...Create user friendly web sites and the spiders will follow. (IMG:style_emoticons/default/spider.gif)

[Randy]

Old timers, or anybody who has read and taken to heart the concepts in Steve Krug's Don't Make Me Think. Or any of the other great usability resources out there for that matter.

There is a method to the madness. (IMG:style_emoticons/default/wink1.gif)

[Jill]

I don't think so, sitemap always helps, whehter it's small sites or ecommerce websites.

In what way?

[lenwood]

I always have and probably always will create human readable site maps because visitors actually use them. But even with those I don't bother trying to link to every single page in a site. I just hit the high points.

Hmm. This prompts more questions for me.

By skipping an XML sitemap and having a user friendly sitemap that links to the high points, you're actually emphasizing the pages that you think are most important? In other words, are you saying that there's value in NOT linking to every bit of content on the site?

Thanks,
Chris

[Randy]

As long as the spiders can get to every page by some means, I've never considered having each and every page in a sitemap necessary.

Frankly, if I were going to do a sitemap to each page of a reasonably sized site in this day and age I'd be far more likely to have those deeper pages appear in some type of css-styled list that was expandable and collapsable. For example, the there would be the Top Level pages/categories that appeared in the map all the time, but if someone clicked on a little Plus image next to one of the cats the sitemap menu would automatically expand to show the lower level pages that appear under that section. Better for users since it wouldn't overwhelm them with 80 links at once, but all of the links would technically still be there for the search engines even if the sub menus were all collapsed.

By skipping an XML sitemap and having a user friendly sitemap that links to the high points, you're actually emphasizing the pages that you think are most important?

It's the old, original way of sculpting PageRank. (IMG:style_emoticons/default/lol.gif)

[Catz]

Old timers, or anybody who has read and taken to heart the concepts in Steve Krug's Don't Make Me Think. Or any of the other great usability resources out there for that matter.

That's assuming usability has even crossed people's minds as anything more than a passing thought.

Graphic designers create websites to show off their "artwork", oblivious to the fact that content is what brings visitors back. They use tiny fonts you need a magnifying glass to see and code their pages in a way that does not let you make the font any larger. (Remember when copyright sized text was only used for the copyright on a page?)

People learn to create a web page with programs like Dreamweaver or Front Page, never having learned a thing about usability, yet consider themselves web designers in no time.

Those same "designers" read a thing or two about SEO, and they are suddenly selling SEO services when they should be studying and learning a lot more about user friendly web design and SEO before expecting others to pay them for these services.

Anyone and everyone who hears you can make a profit at SEO has seemed to decide they should do it too. (IMG:style_emoticons/default/preved.gif)

We all know that's a huge part of why SEO has such a bad reputation...People who should be studying and learning a lot more before even attempting to offer the service to others, are charging money to do something that (many times) they really don't know much about.

[pli]

This is how you can pick the "old timers" out of the crowd. Those with the most experience.

We think through the process and create user friendly web sites that are in turn, spider friendly.

Creating a site map that is useful to your human visitors used to be considered an important part of the web design process, now you see automated lists of links, certainly not created with the human visitor in mind.
I've said it before and I'll say it again...Create user friendly web sites and the spiders will follow. (IMG:style_emoticons/default/spider.gif)

I think that your statement regarding user friendly web site is very valid. However this "old timer" who has been creating sites since 95, has created a site map for every site undertaken over the past 3/4 years or so. I am not saying that it is a "must do", but my observations over the years are that if it seems like a sensible thing to do and reputable seo sites/blogs whatever don't say - don't do it, then do it.

Follow the major se guidelines and if you want to add xml site maps, user site maps, analytics, use webmaster tools or what ever - It MAY help. I believe that doing all those things and plenty of others that folk have suggested on this site and other worthy seo sites (that makes sense) have helped me get excellent results (not just serp results) for my clients.

P