26 replies to this topic

[Jill]

I would think at the very least you would be telling the crawler, hey, here I am, please come and find me soon.

Yes, it seems logical, but it just doesn't work that way.

You have to understand that their add-url links have been severely abused by spammers over the years, so there's really not reason for the engines to put any trust in the info they get through those links.

[Randy]

You'll need to allow a couple of months to get results for even that kind of test Michael. Even then cause and effect can quite possibly become muddled.

The problem with cause and effect these days is that Google is also an Official Domain Registrar. So it's quite possible (and I don't know) that they may be able to tell when a new domain has been registered. Meaning they could see a new domain being registered and simply schedule Googlebot to go visit it in a couple of weeks. And every 2-4 weeks thereafter.

So, in theory at least, the home page could show up as spidered whether one submitted it or not. And whether it was linked to or not.

Since it wouldn't stick without any external links, how could one determine if the domain root got spidered because it had been submitted or because it showed up on some New Domain list that we mere humans are not privy to.

Perhaps there's a way to do it and take the New Domain portion out of the equation if you have enough time before you start adding content. Since the domain is already registered, set up some hosting for it, slap up something that doesn't look too much like a basic placeholder page and let it set for a couple of months. Checking every now and then to see if they've spidered it or not in the log files.

If they don't, but then when you submit it Googlebot comes by, it might actually tell us something regarding whether the New Domain list exists and is used or not.

[Scottie]

OK, I have a few domain names that I haven't developed yet.

I'll set two of them up and just let them sit there. Won't link to them from anywhere. Will submit one to the big 3 and will not submit the other one at all.

Since I'm not ready to work on these yet, we'll see if either or both get visited and/or indexed.

[RyanBlank]

take the New Domain portion out of the equation if you have enough time before you start adding content. Since the domain is already registered, set up some hosting for it, slap up something that doesn't look too much like a basic placeholder page and let it set for a couple of months.

I actually did excatly that on June 9th of this year with a domain i purchased months before and had not yet used. reading your post reminded to configure urchin to analyze the logs for it.. so, i just went ahead and did that and found the only bot to have visited since it's debut on June 9th is this one

strange.. but no google, yahoo or anyone else

<edit>strange that this bot came to the site.. not strange that google and yahoo did not</edit>

Edited by RyanBlank, 16 September 2005 - 03:23 PM.

[Michael Martinez]

I actually did excatly that on June 9th of this year with a domain i purchased months before and had not yet used.  reading your post reminded to configure urchin to analyze the logs for it..  so, i just went ahead and did that and found the only bot to have visited since it's debut on June 9th is this one

strange.. but no google, yahoo or anyone else

But you did not submit the domain to any crawlers, right? That one found the domain on its own.

Randy makes some good points. The new domain I have, when it goes live, should accrue some links fairly quickly. It will be related to one of my well-established areas of expertise and people will find it quickly (it is being set up to meet demand for more output from me).

I may get a window of a week or two before I have to announce it. That may not be enough time for Google to crawl it.

I think I will set up a temporary content page as the root URL and submit it to Google today. At least I'll have a little more time this way, and it won't hurt to have some fluff content spidered (I think -- I probably don't want it to be too fluffy).

[RyanBlank]

But you did not submit the domain to any crawlers, right?

did not submit.. did not add links.. did nada.

[Jill]

OFFTOPIC:

It will be related to one of my well-established areas of expertise Just how many of those do you have, Michael?

[Michael Martinez]

I may get a window of a week or two before I have to announce it.  That may not be enough time for Google to crawl it.

I think I will set up a temporary content page as the root URL and submit it to Google today.  At least I'll have a little more time this way, and it won't hurt to have some fluff content spidered (I think -- I probably don't want it to be too fluffy).

Ain't gonna happen. Since the beginning of this month, that domain has been repeatedly accessed by crawlers like Slurp, SurveyBot/2.3, and T-H-U-N-D-E-R-S-T-O-N-E.

As soon as I put up content, Yahoo! will index it for sure (no idea of what it would rank for or where, of course). But there is no window of invisibility for this domain. At least three crawlers are checking domain names on the basis of changes in WhoIs data.

Jill: Areas of expertise depends on who you talk to, but this domain is Tolkien-related. I'm pretty safe at claiming expertise there.

[ab_websearch_guide]

I actually went to this session. Danny emphasized that you MAY submit to crawlers, but you don't have to. Emphasis on the MAY. Does not really matter one way or t'other.

[charly]

"For years I've been telling our clients that crawler-based engines find your website on their own and submitting is a waste of time." quoted from "TTW" above.

You are right with the above quote, most search engine spiders will always find your site once you are linked in other pages or website. Think of the internet as a spider web, no matter where you are there is a tendency you are connected.

The spider bots visit pages looking for fresh link to index.

It is important you place your website link to most site(s) the spider will likely visit like the directories, forums or older internet websites.

At least this work mostly in major search engines than the very small ones.

[Scottie]

The test is in progress- 4 domains, each with a home page describing what will be on that site eventually have been set up.

I submitted one manually to Google, Yahoo, and MSN on Sunday. No response yet.

I submitted one through a free submission service. Again, no spiders have visited but I do have some interesting email starting to arrive on that domain.

The other two are just sitting there. No links, no submissions. All these domains were registered about 3 months ago.

Will keep you posted, but so far all I'm getting is spam mail.

Welcome to the forum, charly!

[Scottie]

Update:

Here is what I did-

Site manually submitted to Google, Yahoo, MSN. Now showing one page indexed in Yahoo, MSN. Nothing in Google. Googlebot has not come to the site, slurp and MSNBot came once. Page does not show up for it's own title tag in either engine.

Site submitted to Submit Express: Got a response back from the Jayde directory to "correct my submission". I selected a category and submitted. Now I get WebProNews every couple of days in that account. No spiders have visited, no pages in any index yet.

Site that has content, but not submitted. No spiders have visited. Not in any indexes.

Last site- has content but not submitted. Will be adding a few links to it this week (but not submitting) and will let you know what happens.