Skip navigation
newsletter signup

High Rankings Advisor: Blackballed from Google - Issue No. 017

July 3, 2002
________________________________________________________

~~~IN TODAY'S ADVISOR~~~

*Introductory Comments:
---->   Keeping Cool

*Search Engine Marketing:
---->   Blackballed from Google
---->   Keyword-rich Domain Names Feedback

*This Week's Sponsor:
---->   LinkSurvey Link-Popularity Software

*Other SEO News:
---->   Terra Lycos Launches Lycos 6.0
---->   Deceptive Search Engine Advertising

*Stuff You Might Like
---->   Selling Subscriptions to Internet Content

*Advisor Wrap-Up:
---->   I'm a Yankee Doodle Dandy
________________________________________________________

~~~Introductory Comments~~~

I've got a fairly long issue for you today.  I'm avoiding the heat
wave by holing up in my air-conditioned office and just writing,
writing and writing!  I'm sure many of my American readers are on
vacation this week, and I'm expecting a huge amount of out-of-office
replies at any moment now.  Hopefully you'll at least read this when
you get back.  There's lots of good stuff here, including feedback to
last week's keyword-rich-domain rant and my rebuttal.  So without
further ado, let's move on to the good stuff! - Jill


~~~Search Engine Marketing Issues~~~

++Blackballed from Google++

From: Bob M.

Dear Jill,

Thanks for your great newsletter.  It's one of the few, perhaps the
only one, of the many newsletters that I subscribe to that I find
regularly informative and useful.  I especially enjoy your informative
rants, so perhaps I'll supply justification for you to climb up on
your soapbox and wax eloquent.

I'm relatively new to search engine optimization.  Earlier this year,
when I knew less than I do now, and at the request of upper
management, I submitted our site (already listed) to Google under a
different URL. I was wary of this strategy at the time, and have
become more so the more
I have learned.

I'm not surprised that a search of Google for the alias URL produces
no results.  But I'm also concerned whether Google may have taken
punitive action regarding our existing, established site/URL.  One
pretty good hint is that one keyword that had produced a #4 Google
rank last November now shows nothing for us in the first 20 pages.

Two other bits of information complicate drawing any causal inference:
first, we completely redesigned the site in February this year.  (I've
been careful to make the content keyword rich and appropriate.)
Second, I know Google gets search results from Open Directory in
addition to the information gained from their own crawlers.  Open's
volunteer editor policy means the updated information we sent them in
February has yet to show up in our listing with them (or, for that
matter, in Google's).  Nonetheless, our keyword ranks in Open are very
good, in stark contrast to Google's.

My primary questions are whether our Google rankings are being
suppressed in response to our misguided attempt at a duplicate
listing, and, if so, how we can atone for our sins and restore our
good standing with them.

Thanks for your weekly information in general, and any help in
particular.

Bob

++Jill's Response++

(Note: Bob didn't want his site to be mentioned in the newsletter for
obvious reasons, but it was supplied in his original email.)

When I get these kinds of questions, the first thing I do is check the
site with my Google Toolbar turned on, so I can see if the PageRank
graph is grayed out or at zero.  If Google has imposed a penalty on a
site, it's usually evident by looking at the PageRank.  (For more info
on PageRank and the Google Toolbar, please read my PageRank Summary
here: <http://www.rankwrite.com/archives/issue070.htm#seo>.
Gee...lots of Rank Write references today!)

So I plugged Bob's site into IE and saw that it had a respectable
PageRank of 5, which indicates that there's no penalty involved.
Next, I checked Google's cache of the page to see if they were showing
a blank page, or something other than the current site.  Strangely
enough, Google had no record of the page in its cache.  So I checked
the backward links, because usually if it's not in the cache, there
will also be no backward links.  However, there were *a lot* of
backward links.  So things seemed stranger by the minute.  The site
was indeed listed in DMOZ as Bob had stated, and also in Yahoo!.  It's
got backward links and a good PageRank, so what could be the problem?

It seemed to me that for some reason Google must not have been able to
spider the site.  My first thought was that the server may have been
down when the Googlebot came a-crawlin'.  But then something else hit
me.  Perhaps Googlebot *couldn't* spider the site. Perhaps it was
excluded from crawling the site through the robots.txt file.

For those who don't know what this is, it's a simple text file that
you can put on your server to exclude search engine crawlers from
accessing certain pages or directories of your site.  For instance, if
you have password-protected directories on your site with info that
you don't want the general public to get their hands on, you might
exclude crawlers using this file.  (For more information on this,
please see: <http://www.robotstxt.org/>.)

So the next thing I needed to do was check out Bob's robots.txt file.
(To do that, you simply type in the domain name followed by
"/robots.txt" into your browser, e.g., bobsdomain.com/robots.txt.)

Here's what I found there:

User-agent: *
Disallow: /
Disallow: /Admin
Disallow: /Appraisal
Disallow: /Content
Disallow: /Custom
Disallow: /Images
Disallow: /Logon

Aha!  There was the answer I suspected!  Someone in Bob's organization
had put up a robots.txt file that excluded ALL search engines from
indexing ALL parts of his site!  (To be sure I was reading the file
correctly, I checked with a techie friend, who confirmed my
suspicions.)

The moral of this story is that if your site is not showing up in any
given engine, the chances are that you are *not* banned.  It's
actually very, very rare for engines to ban or penalize sites.  You
have to be doing some pretty nasty things for that to happen.  It's
extremely rare to be banned by mistake or simply because you did
something that you didn't know would be considered spam.  Those that
get banned for real almost always know *exactly* what they did wrong.

So don't just assume that you're banned if your site is missing.  Do
some detective work and find out the real reason, then fix it!  I've
seen other instances where the problem had to do with misconfigured
servers and IPs and other things like that.  Sometimes it's as simple
as your site being down when the bot tried to visit it.  Just remember
that you're probably *not* banned.  If you think you did something
that the engine might consider spam, then fix it and wait for the next
crawl.

If you never do anything even remotely shady, you won't have to worry,
now will you?


____________________________________________adv.

LOOKING FOR HIGH-QUALITY LINK PARTNERS?
____________________________________________

LinkSurvey link-popularity software is the tool you need.

Check the link pop. of a whole bunch of related sites all at once.
Our user-friendly software saves you tons of time & aggravation!

Search through all the major search engines in one fell swoop,
and quickly create your own detailed link-popularity reports.

Free trial and no-risk money-back guarantee!  Learn more
now at: <http://www.antssoft.com/linksurvey/index.htm>.
_____________________________________________


++Keyword-rich Domain Names Feedback++

From: Chris Beasley

Sorry Jill, you've got some misconceptions about keyword rich domain
names.

First of all rarely do I see people advocate them for search engines,
however people do tend to refer to any search service as a search
engine, when this is simply not so.  They need to differentiate
between search engines, directories, and simple portals.

However a keyword rich URL still helps for search engines, you just
haven't thought things through far enough.

You mention this:

>>Some rank highly simply because there are keywords in the hyperlinks
pointing to the site.  Google is especially susceptible to this
phenomenon, which has been dubbed "Google Bombing" in the Web Blog
world.  If enough sites link to a site using the specific keyword
phrase in the hyperlink, it can have a dramatic effect on rankings. <<

Which shows that you know how important anchor text is with link
popularity.

However what you do not realize is that almost all links take two
forms and they either have the site title, or the URL, as the anchor
text.  Obviously, if you have keywords in your site title (not your
title tags, your actual title), or in your URL then this will help
you.  So its not that the keyword rich URLs are helping directly, its
that they are helping indirectly.

On the topic of directories you mention:

>>With directories such as Yahoo!, it may appear as if keywords in the
domain name make a big difference to rankings.  After all, the
directories are not spidering the words on your pages and don't have
much to go on to determine your position.  But you know what?  My very
unscientific research shows that in actuality, keywords in the Yahoo!
title are the more likely reason for top rankings.<<

Obviously why even publish that if its so unscientific.  I could just
as easily claim that the URL was weighted more than the title.  As for
how much each portion is weighted?  I'm guessing its about equal,
however the fact of the matter is it is very easy to provide proof
that Yahoo, and other directories, take URLs into their ranking
algorithm.

For instance, check this out:
<http://search.yahoo.com/bin/search?p=webmaster+resources>

Notice those bold words, those are words that Yahoo! counts in it's
ranking algorithm.  You will notice that it counts titles,
descriptions, URLs, and category names.  Additionally Yahoo! probably
uses click-popularity and they most likely have a hidden field that
editors can use to rank sites.  LookSmart has such a field after all.

Another thing I take issue with is this claim:

>>Just to be sure I wasn't leading you down the wrong path, I asked
Tom Wilde, General Manager of Search Services at Lycos, if domain
names factored into their ranking algorithm.  He told me that they are
a factor in the Lycos algorithm, but since they're so open to being
abused, they're a very small factor.  He confirmed that the Title tag
and the body text copy were given much more weight than the domain
name.<<

How is the title tag any less abuseable than the domain name?  At
least domains cost money and if you have a good one you're either
lucky or you paid a decent amount of money for it.  That's more of a
question for him, not you though, since you're just repeating what he
said.  At least he supports the use of hyphens, that parsers often
require punctuation to parse keywords is also verifiable simply by
looking through results at Yahoo or DMOZ.  For instance Yahoo! will
parse the first keyword in a run on word, so in the word "carcity" it
would parse "car" but not "city." DMOZ would parse neither.

The biggest reason to have keywords in your site name and in your
domain name is for link popularity.  I'm sure you know how important
anchor text is, and having keywords in the two most common ways people
link to you is a big help.  I tend to use keyword rich names and
domain names for all my sites, and I've gotten #1 listings on every
major search engine (except Inktomi - but they're dying anyways)
thanks in part to that.

For businesses or e-commerce sites I would agree with you that a good
name is better.  But for content sites, where you need so many
visitors to make a buck, I'd sacrifice brandability for traffic.  You
really have to be honest with yourself in regards to branding too.
Branding is expensive and unless you've got tens of thousands or even
millions of dollars to spend on advertising in most cases you will not
brand your name.  Buying a unique name does not make you branded;
branding requires gobs of money or gobs of luck. Most people would be
better off if they just admit that it is not going to happen and
instead they work on building traffic.

Chris Beasley
http://www.sitepoint.com/articlelist.php/82


++Jill's Response++

Thanks for sending in your thoughts on this issue, Chris.  I always
appreciate feedback, and your comments are well taken.

A few others also wrote in to express their disagreement with me, but
not as many as I thought would.  The notion of using keyword-rich
domain names as part of your SEO strategy is so ingrained in
everyone's heads that they are not willing to give it up without a
fight!  Many have based their search engine strategies on purchasing
multiple keyword-rich domains.  Downplaying its importance is
obviously not going to sit well with everybody.

I do need to comment on what Chris and others have pointed out.  What
I was talking about in my article was the *direct* influence that a
domain name has with the search engine's ranking algorithm.  I
maintain that the engines do not place much emphasis on the words in
the domain name itself.  But I do agree that there can be an
*indirect* benefit from these domains.  That's because these sites are
also using keywords in the site name, the Title, the copy and the
links pointing to the site.

So yes...of course there is an indirect benefit that comes from having
your keywords in your domain name.  It becomes part of your site's
identity and how other sites refer you.  It encourages those linking
to you to use those same keywords in your hyperlink.

However, you'll get the same effect simply by naming your company
something that uses the keywords.  For instance, let's take my Rank
Write site.  I noticed a long time ago that I was ranking very high
for the single word "rank."  I never optimized for such a broad term,
yet the site is in the top-10 in nearly every engine and directory.  I
had originally assumed it must be because the word was in the domain
name.  In fact, I mentioned this once in RW issue 060 from August 2001
<http://www.rankwrite.com/archives/issue060.htm>. (The question
concerned whether to hyphenate words in the domain name.)

Here's what I said:

"Although I do believe that the search engines see dashes as a space,
which could theoretically be helpful, I also have a gut feeling that
they can also separate words in a domain name.  I say this because
I've seen our RankWrite.com site rank high for the one-word 'rank' in
a few engines and I think it might be partially attributed to the fact
that the word 'rank' is in our domain name.  Again, I haven't studied
this; it's just something I've noticed."

Well, that was nearly a year ago.  After speaking with Tom Wilde from
Lycos last week, and after hearing others' opinions on the matter in
my forum hangout (Ihelpyou), I now believe that having the word "Rank"
in the domain name was not what was causing my top positions for that
word, but having the word "Rank" in the name of the site, Rank Write.
If you take a look at the backward links to the Rank Write site, a
good portion of them use the name "Rank Write" in the hyperlink
pointing to it.  But even if the domain name was
JillandHeathersRantingPlatform.com, as long as the site was still
named Rank Write, the links to the site would be using the word
"Rank."

The thing is, it's unusual to have a site named one thing and not have
its domain name match up.  If I used that other domain name, I
probably would also name the site "Jill and Heather's Ranting
Platform."  Then perhaps the site would rank highly for "Ranting"
instead of "Rank."  But it wouldn't be a direct result of the domain
name.  It would be an indirect result.

Certainly if all else is equal on two sites (and this is rare), the
domain name could give one site the edge; I've never disputed that.
But it's really all the other indirect factors that matter much more
than the actual words in the domain name.

The fact of the matter is that all the evidence that anyone has sent
me to show that keyword-rich domains DIRECTLY affect the rankings in
the spidering search engines can *always* be explained through other
means.  Until I see some direct evidence where *only* the domain name
is causing the high rankings, I will stick with what I've been saying
all along.

If you're starting a new company, then by all means, think of a
keyword-rich name for it, and purchase the matching domain.  I
absolutely believe that's a good idea.  (Not something with more than
two keywords in it, however.)  If you have an existing company that
already has a name without keywords in it, please don't sweat it.
Don't go nuts and change your company name.  Don't purchase tons of
extra domain names "just in case."  Just do what I always say to do:
Make your site the best it can be by writing about your products or
services using the words that real people use.

Thanks to all those at the Ihelpyou forums who helped me to clarify my
stance on this matter:
<http://www.ihelpyouservices.com/forums/showthread.php?s=&threadid=373
1>.

~~~Other SEO News~~~

++Terra Lycos Launches Lycos 6.0++

Lycos has added lots of new features, including the integration of
FAST's entire database (apparently the largest in the world), support
for PDF searches and an extremely up-to-date news search.

This is all great news for Lycos, but they will have to make a
concerted effort to get the general American public to start using
their search engine.  (I hear they're big in the UK.)  All the
features in the world don't matter much to site owners and SEOs if
people aren't searching there.  It will be an uphill climb to get
Google users to switch to Lycos, that's for sure.  As I've said many
times before, the title of Most Popular Search Engine is Google's to
lose.  They will have to screw up badly to get people to go elsewhere.
And even if they do, it will still take a long time for people to
switch.  The problem for search engines like Lycos is that there are
no signs of an imminent Google screw-up.

At any rate, good luck with the new launch, Lycos.  I like the look
and ease of use.  But I'm still using Google!  (Do you ever wonder if
the bigwigs at the other engines also use Google? <grin>)

Read more about Lycos 6.0 here:
<http://www.terralycos.com/press/pr_07_01e_02.html> and see their new
interface here: <http://www.lycos.com>.


++Deceptive Search Engine Advertising++

Remember last year around this time when Ralph Nader's "Commercial
Alert" filed a complaint against the search engines for deceptive
advertising?  They claimed that search engine ads (such as
pay-per-click links) were not always clearly labeled as such, and
therefore the searching public was being misled.  (You can read the
original complaint here:
<http://www.commercialalert.org/index.php?category_id=1&subcategory_id
=24&article_id=33>.)

Well, the US Federal Trade Commission (FTC) has finally had a chance
to respond to the complaint.  You can read it in PDF format here:
<http://www.commercialalert.org/PDFs/ftcresponse.pdf>.  It's actually
a very interesting read.

Many of the search engines and directories had already started to do a
better job of labeling their ads as "sponsored results"; however, it's
still not always obvious to the general searching public.  It will be
interesting to see if the FTC's response has any effect on how the
results are labeled in the coming months.

The FTC also discussed pay-per-inclusion (PPI) results in their
letter.  These would include Inktomi listings that were paid for
through companies like PositionTech.  I can't imagine how they could
label those types of sites, nor am I sure that they need to be
labeled.  At any rate, we'll see what happens!


~~~Stuff You Might Like~~~

++Selling Subscriptions to Internet Content++

This has nothing to do with search engine optimization, but last
weekend I read the entire 253-page transcript from ContentBiz's "2nd
Annual Selling Subscriptions to Internet Content Summit" which was
held in NYC on May 21, 2002.

To tell you the truth, I assumed a word-for-word transcript of the
entire event was going to be extremely boring.  Much to my surprise,
this very fat, spiral-bound transcript kept me enthralled the entire
weekend.  To be sure, I did skip through certain parts of some
speakers' presentations, but most of them were riveting.  I could
actually picture the speakers up there at the podium giving their
spiels, and they were very good!

The subject matter was very interesting to me because I have had
fleeting thoughts about making this very newsletter a paid
subscription.  From the feedback I get, and the growth in subscriber
numbers each week, I do think it would be a viable option at some
point.  Reading the transcript regarding ten companies who have
successfully made the transition was a great learning experience.  The
company reps shared all their secrets, including the mistakes they
made along the way.  One could definitely learn what to do and what
not to do when making such a huge change in business models.  The
speakers also gave some great advice on what kinds of content could
successfully move to a paid model.  (I do think mine would fit...but
don't worry, I don't have any plans in the works -- yet!  As long as
you guys keep sending me chocolate and purchasing some of the stuff I
recommend, I'm cool with the status quo.)

If you have great content that you're currently giving away for free
but would like to start charging for, then this transcript is a
*must-read*.  Please note that it's not cheap.  It'll set you back
$199, and it's only available in printed form (shipped Priority Mail).
However, if you're seriously considering making the move, you'll
easily make your $199 back by not avoiding the costly mistakes of
those that came before you!

To learn more and/or order your copy, please visit:
<http://www.site-pros.com/contentbiz>.


~~~Advisor Wrap-Up~~~

Happy birthday to me!  (Oh, and to America also!).  Yep, I'm a July
4th baby.  There's nothing like the special feeling I get when those
fireworks shoot off every year in my honor.  What's that you say? The
birth of our country? Shhhh...don't shatter my illusions!

To all those who get my birthday off from work, enjoy it!  I'll be
doing my favorite thing -- working!  - Jill
 
Email a FriendPrintRSS