Skip navigation
SEO Website Audit

High Rankings Advisor: How deep can spiders crawl - Issue No. 050

April 9, 2003


*Introductory Comments:
---->   All for the Price of Chocolate

*Search Engine Marketing:
---->   How Many Levels Deep?

*This Week's Sponsor:
---->   Award-winning IBP Software
---->   Nitty-gritty of Writing for the Search Engines

*Guest Article:
---->   Getting Organized with Usability Checklists

*Other SEO News:
---->   Atlanta Half-day SEO Basics Seminar
---->   New Yahoo Search

*This Week's Sound Advice:
---->   Misconceptions About Google PageRank

*Advisor Wrap-up:
---->   Anybody Know Mother Nature's Number?

~~~Introductory Comments~~~

Hey guys!  It's the 50th issue!  Got more good stuff for you today.  I
finally got around to answering a question that I see a lot in the
forums, i.e., can pages many levels down in the directory structure
get spidered?  Plus, we've got the second installment to Kim's
usability series of articles.  Good stuff.  All this just for the
price of a little chocolate.  What? You haven't sent me your chocolate
yet?  Okay, you can't read any further then. ;-)

For the rest of you who've been good and sent your chocolate...on to
the good stuff! - Jill

~~~Search Engine Marketing Issues~~~

++How Many Levels Deep?++

From: Jeff Laurie

Jill --

Congratulations on the spectacular success you've achieved in the SEO

I understand that spiders only go to the second level of a site.
Unfortunately, all of our 413 previous newsletter issues are on the
third level of the site.  If I put a page on the second level -- an
archive with links to the individual issues -- would spiders follow
and index the links to the third level?

Thanks a lot in advance.  Keep up the great work.


~~~Jill's Reply~~~

Hi Jeff,

Actually, the spiders will go to any depth of a site, not just the
second level.  However, they don't place as much emphasis on pages
within the site that take more than a couple of clicks to get to from
the main page.  It's not the physical directory structure that slows
the spiders down, but really the number of clicks it takes to get to
the pages.

In other words, say you have a page that's four-levels deep in the
directory structure like this:  Let's also say that
there's no link to that page from the home page.  Let's go even
further and say there's no link to that page from the next level of
pages (which would be common for a page buried four-directory levels
deep).  Let's say that people don't generally find that page (when
starting at the home page) until they've clicked through from three
other levels of the site.  That page is pretty hard to find!

It's the same for the search engine spiders.  Very often they take a
really fast crawl through your site and don't bother to fetch pages
that are hard to find.  Sure, they may get to those pages eventually
when they're in the mood for a deep crawl, but it won't happen very

Now let's take that same page and put a link to it from the home page
of the site.  It's still physically four directory levels deep, but it
only takes one click for a person to find it.  With that scenario, the
search engine spiders will also be able to easily find it!

Now, there are of course exceptions to this rule.  If other sites
happen to link to your deep page from one of their top-level pages, it
can have a similar effect as if you did.  But you can't control how
others link to you, so don't leave it up to chance!  If you have a
deep page that changes often, and you want to ensure that the latest
version is apt to be spidered and indexed, then make it easy to find.

Your best bet to ensure all your content gets spidered is to create a
sitemap that lists all your pages, and then link it visibly and
directly from the home page of your site.

Hope this helps!



What are the top 10 ranking sites doing that you're not?

Award-winning IBP Software quickly provides you with the answers.

Choose your search engine, plug in your URL and keywords, then sit
back while IBP analyzes more than 77 ranking items that the top sites
are using and you're not!  Follow the easy-to-understand optimization
report to place your site at the top of the engines where it belongs!

Learn more here:  <>.

~~~Guest Article~~~

++Getting Organized with Usability Checklists++

The second part of our "Being Tops with Your Users and the Search
Engines" series by Kim Krause has to do with the organization of your
site.  In order to design a fully functioning site that is highly
usable by all, it's got to be organized.  To make it easier for us
scatterbrained individuals who wouldn't know organization if it
smacked us between the eyes, Kim uses the analogy of producing a play!

Here's what she recently told me about this:  "The way I came to view
Quality Assurance User Interface and Usability testing for websites
was to visualize each step as if producing a play.  There is a plot,
set, scenes, characters and audience."

So let's sit back and relax while Kim shares her "website design play"
and explains how she puts it all together through the magic of

Being Tops with Your Users and the Search Engines
Part 2 - Getting Organized With Usability Checklists
By Kimberly Kopp Krause

The Plot - What Your Site Is About

Your plot is the story you want to tell about your business. To create
it, take the goals and objectives from your "Business Requirements"
document (see Part 1 </issue049.htm#guest>)
and write a short paragraph describing what your website is going to
be about. Don't be afraid to add some drama to it to make it
interesting.  Here's an example:

"Health Wise Products Inc., well established in Villageplace, Idaho,
launched several products and services for its medical practice
clients.  Due to BigShot Health Company Inc.'s recent decision to move
their headquarters to the same town, Health Wise CEO Marvin Smart
introduced a new product line for patients. In an extraordinary move,
he also added a pet psychic to provide online therapeutic services for
pet owners using online software for purchases."

This plot hints at long-term goals, target market, competition, target
area (local vs. global site), reputation and the need to build an
intuitive website to fulfill software requirements for online
purchasing.  All this is in one tidy paragraph! Brainstorming sessions
for developing plots is not only fun, but also it helps everyone to
start thinking creatively about the project.

The Characters - Developing a User Character

A user character is a person you develop and imagine as someone who is
likely to use your web site. These user characters help you determine
how usable your site because they may use it in ways you may not have
thought of when being yourself.

Developing user characters is similar to creating the characters for a
play.  Characters often develop further as the story unfolds.  Your
user persona will also morph, depending on the tasks to be performed.

To come up with a user character you'll need to ask these questions:
* What are the demographics of your user (e.g., gender, age, income)?
* What is their main role in life (e.g., single parent, student,
military, PhD, youth, hermit, CEO)?
* Are they computer-literate (e.g., define all expected computer
experience levels, including operating systems, browser types, Mac or
PC, etc.)?
* What motivates them in life and what will motivate them on your
website (e.g., if they're a purchasing agent, what does your site
provide to make their online bargain hunting easier)?
* What are the main tasks they will perform on your site?
* What are their goals?
* Do they have special needs?
* Are they more apt to research via search engines or through print?
(e.g., are they brand-conscious, can they spell, do they know how to
search with keywords to get good results?)

User characters can be highly specific, all the way down to what they
had for breakfast, if they missed their train that day and whether
they are multi-task oriented or suffer from Attention Deficit

If you compile a number of personality types they can actually "tell"
you what they need and want from your site if you take a moment to get
into their heads.

The Setting - Where It All Takes Place

This is where you decide the look and feel of your website.  You may
think this is the easiest part, but in fact, this area demands strict
attention to detail and draws from many related fields.  You'll need
search engine optimization, graphics, programming, HTML, marketing and
security skills, as well as knowledge of hardware and software, the
Internet, design standards, copywriting and even a bit of Internet

To review your website's setting, you'll need a checklist of areas to

* Stickiness: what attracts users and keeps them coming back?
* Content: is it authentic, credible and readable?
* User interface: what are the heuristics, colors, fonts and design
* Customer service: is there a place for feedback? Are there
incentives?  Are pages printable?
* Security: is the data and the server secure? Is there a privacy
policy in place?
* Form design: can international users fill out all form fields?
* Accessibility: can special-needs visitors access all parts of the
* CYA (Cover Your Asset): Is the product one that won't crash the
users' computers?  Is the name trademarked and are patents registered?

Design your test cases based on your "business and functional
requirements" to make sure each of them were met.

Scenes - Navigation

Every story has certain shared elements, such as a preface,
introduction, climax and conclusion -- with scenes that twist and turn
in between.  With your website, the story is told by navigational
links that take the user to what they need to know and do.

You can create your scenes by designing hubs at the top level, such as
"About Us" and "Products and Services."  Templates that change
depending on the different category themes designate a new "scene" in
your layout.

In a play, each scene has acts; so do the hubs of your website. For
the "About Us" hub, Act One might be "Press Releases," Act Two might
be "White Papers" and so on.  The scene doesn't change but the topic
switches and your character (the user) is sucked deeper into your
story (the website).  Diagramming your navigation through storyboards
or even just on paper will help you stay organized and focused.

The Audience - Who Is Paying the Bill

The audience includes the owner of the website, the stakeholders,
along with competitors, search engines, and other sites that may want
to link to yours.

Well-run companies don't proceed on projects until their stakeholders
sign off on the business and functional requirements at each stage of
the development cycle.  If you're a solo designer or small team, try
to use that same strict focus on organization, planning and attention
to detail -- and you will be rewarded with a dream website that sells!

Here are some free checklists:

Directory submission, homepage elements and website promotion
checklists: <>.

User Character (Persona) Checklists:
* Designing for the Multiple Personalities of Users
* Personas: Matching a Design to the Users' Goals
<>, and
* Perfecting Your Personas by Kim Goodwin, Director of Design

Next Week - How To Obtain User Feedback

Kimberly Kopp Krause
Cre8asite Forums:


"Keywords in your visible page copy are the KEY to High Rankings!"

You can stick 'em in the Title tag and in the Meta tags.
You can even stick 'em in the Alt tags and hyperlinks.

But you won't see high rankings unless you stick 'em in the copy.

Find out how in Jill's "Nitty-gritty of Writing for the Search

~~~Other SEO News~~~

++Jill's Half-day Atlanta SEO Basics Seminar++

Register here: </register.htm>.

Okay guys, we're getting close to the cut-off point for the early-bird
$50 off registration.  If you want to meet me in Atlanta on May 16 and
hear me spill my guts on everything that is SEO, it's time to
register.  After April 16th the price is set to go up to $299.  I know
I'm worth that extra $50, but why not save it for something else?  The
special $89 hotel rate also expires on April 16.

Let's put it this way.  There are not many ways to learn everything
there is to know about search engine optimization for only $249.
There just aren't.  Even a  one-hour phone consultation with me runs
up to $600, and I can't cover everything on the phone in an hour.
There's just no way.

But I do cover *everything* in this seminar.  I know it seems hard to
believe, but if you are a beginner or advanced beginner trying to get
high rankings for your site, then you seriously need to come to this
seminar.  Don't worry about being overwhelmed.  That's why I'm keeping
it to just a half-day.  It's long enough to cover everything you
really need to know, but not so long as to confuse you.  I'm extremely
confident that if you live within driving distance of Atlanta, you
will absolutely, positively get your money's worth from this seminar.
(And even if you gotta fly in, it's still gonna be worth it!)

You can learn what's on the agenda here:

Plus, don't forget about lunch!  We still have spots left for the
exclusive luncheon after the seminar.  That's where you can ask me all
questions you weren't able to ask in the class itself for whatever
reason, and just have fun chatting with me and your peers.

Oh yeah, and here's a little secret I've been saving.  I've got some
special guest stars who will be on hand.  By now, you've all heard
about my friend and SEO copywriter, Karon Thackston.  Well, she's
gonna be there, and I'm sure I can convince her to answer your
copywriting questions if I ask her nicely.  Plus, my other friend and
link building specialist, Debra Mastaler, also just happens to be
coming.  I imagine we could eek out a few link-building tips from her
too, if you're really good!  I'm a bit behind schedule in lining up
sponsors for the event, but you can most likely count on some search
engine and/or product reps to be available also.  I'll get back to you
with more on that next week.

Here's that registration link again:

++New Yahoo Search++

Have you fooled around with the new "Yahoo! Search" yet?  What d'ya

Before discussing the particulars of Yahoo's changes, let me answer
the question we've all been waiting months to find out, i.e., is Yahoo
gonna dump Google now that they've purchased Inktomi?

And the answer -- straight from the horse's mouth (to me in a phone
conversation) -- is that they have no immediate plans to get rid of
Google.  They *are* working on ways to integrate Inktomi into various
aspects of Yahoo, though.

I was one of the lucky few who got a sneak preview last week, but I
had to keep my mouth shut until now because it was under "embargo"
till this week.  I have to admit that I cheated.  I told my husband
about some of the features on Sunday cuz I was so excited!
Shhhhh...don't tell Yahoo or I won't be on their A-list anymore!

If you're one of those millions of people who have a Yahoo ID (is
there anyone left who doesn't?), you probably spend some of your day
at Yahoo already.  I imagine that most of you use some of their many
features such as their email or their news or perhaps like me you play
their games or check the local weather.  Do you then switch to Google
when you need to search for something, or do you stick with Yahoo?

I used to switch to Google most of the time.  I like their cache
feature and I really can't stand Yahoo's spammy directory results.
But now, they've actually added the missing cache feature, plus a
whole bunch of other cool stuff!

Take the weather, for instance.  Every time I wanted to check my local
weather before, I had to search through the links at the top of Yahoo
to find the one that said "weather."  (I know, I could have bookmarked
the specific weather page, but I don't check the weather all *that*
much and it seemed like too much trouble.)  Now, if I want to check
the weather, I just type into the search box "weather ashland ma" and
up pops the day's weather forecast!  Pretty neat.  I love shortcuts
like that.

Do you have a Yahoo email account?  Just type in "mail!" (with the
exclamation point as that's their shortcut symbol), and bingo --
you're brought directly to your email page.  Want to play a nice game
of "Dynomite"?  Type in "games!" and you'll be brought to their main
games page.  (I tried "dynomite!" but it didn't bring me directly to
the game.)

They've got a brand-new search page to go along with all of this.  You
can go directly to it here: <>.  It's very
sparse (a good thing) and simply gives you a search box, plus the
choice of Web, Directory, News, Yellow Pages, Images and Maps.  If you
haven't checked it out yet, you may want to take their grand tour
here: <>.  To learn about the various
shortcuts available, check out this page:
<>.  The
dictionary one is really cool too!

~~~This Week's Sound Advice~~~

++Misconceptions About Google PageRank++


Don't forget, you can still purchase my audio CD of the entire "Sound
Advice on Search Engine Optimization" series for the introductory
price of only $29.95 (includes the bonus written transcript in a PDF
file).  Visit the Sound Advice site here:
</soundadvice> for more info.

~~~Advisor Wrap-up~~~ I know that things are crazy -- we've got snow again!
Remember back in October I told you we had snow and even took a photo
of our prettily colored fall trees covered with snow?
</fallsnow.jpg> Well, it hasn't stopped
since then!  October-April.  Sheesh.  At this rate, it will be snowing
on my birthday, July 4th.  Now *that* will be a first!

For my parents and sister, who live in Hawaii, I'm blowing a giant
raspberry at you!

Catch you next time! - Jill

Email a FriendPrintRSS