22 replies to this topic

[electricfan]

Hi everyone..
Just found your forum last week so I thought I'd just jump in.

I've seen that many SE's can take up to 6 weeks for your site to show up.

Question:
If I want to work on my SEO, which engine should I go for knowing it would be the faster turnaround.

Thanks,
Electric Fan

[compar]

I've seen that many SE's can take up to 6 weeks for your site to show up.

Question:
If I want to work on my SEO, which engine should I go for knowing it would be the faster turnaround.

Hi Electric Fan and welcome to the Forum,

The answer to that question is Google. Google crawls at least once in every thirty. So on average they should find your site -- assuming there are links to it from other indexed pages -- on average within 15 days. I have personally seem them find sites in 24 hours.

Now they won't necessarily give you a page rank in that time period or update their cache copy, but they will find you and list you. We put up a site to elect ex Governor Ryan of Illinois for the Nobel Peace Prize at 5:00 on Christmas Eve. Google had the site indexed on Boxing Day. Pretty amazing stuff!

The other side of the question is that even if they were slow you should still put your effort on optimizing for Google. They return 76% of all the searches on the Internet. Google is clearly in first place and there is nobody in second or third.

[JamesW]

From this result, is it possible to be more specific and say that Google indexes from the 25th - 30th of each month? Or is this perhaps a step too far?

Cheers,

James

[Mel]

Hi James;
sorry but Google now appears to be moving towards a more constant spidering and updating of the index, but it looks to me like they are still not quite keeping up with getting all the back links counted. so its hard to estimate a date that you will see your results.

While Google does have the lions share of the search traffic (See Bob I can agree with you :stunned: ) if you want to see fast results you might want to look at paid submission to say Inktomi, which will let you tweak your pages in MSN in a week or two.

[TheGreatDane]

I agree with Mel. Especially if you are a newbie to SEO (like me), it's nice to get a fast indication as to whether you are on the right track. I have for instance made a paid submission to Inktomi on Wednesday this week, and Friday I was listed (they guarantee within 72 hours).

The good thing is, that the indexing will be refreshed every 48 hours, so if you make some changes, you can very quickly see if - and which - difference that made ( as opposed to Google's indexing response of several weeks, which is typical). The submission fee is very acceptable I find, at least if you are talking few pages.

[Jill]

Google has no set dates when they will spider your site. Some they spider daily, some monthly.

Jill

[compar]

Google has no set dates when they will spider your site.  Some they spider daily, some monthly.

Jill

To answer several of the questions above, here is how I understand Google works. Their bots are working all the time. The claim they will crawl the entire web at least once every 30 days. And as Jill says more popular sites are probably crawled more than once. So if you put up a site today your worst case scenario is that you might not be crawled for 30 day. But by simple mathematics that says that the average time is 15 days.

Now if you have substantial backlinks on the day you launch your site the odds are that Google will find you via one of these backlinks very quickly. Because if they happen to be crawling one of the site with a link to you on day one they will follow that link and find you. So the more backlinks you have the sooner they are liable to find you.

BTW this has been said a million times but I still see people who don't seem to understand it, "The only way Google will find a new site is via its backlinks". If you don't have any backlinks you won't get found.

[JamesW]

What is the technical definition of a 'back link'? A web site linking back to you, a reciprocal listing?

Cheers,

James

[Ron Carnell]

But by simple mathematics that says that the average time is 15 days.

Unfortunately, Bob, I doubt the math is that simple.

We're not talking about the likelihood of a single event, but of a specific and ordered sequence of events, some of which have to be guessed. The first thing that has to happen is that the spider finds a link to the new site. THAT should happen within thirty days, or, as you inferred, Bob, on an "average" schedule of fifteen days. If Googlebot immediately followed the link and indexed the new site, things could stay simple. Indication are, however, that the link is instead set aside, probably for DNS resolution. That's when things start to get more than a little fuzzy.

How soon will Google follow the link it found? Some have speculated that if the link was on a PR9 site, it would be given higher priority than if it were found on a PR2 site. Maybe. I don't have any PR9 sites, so your guess is as good as mine.

When the link is eventually followed, the first thing that will happen is a request for the robots.txt file. In my admittedly limited experience, if that file is found, the spider will take it and run back home. It does nothing until your exclusions are digested (parsed and probably stored in the database). When the spider again returns (or if a robots.txt is not found on the previous visit), it will grab the domain's default page. And, again, run right back home. Only when that page is digested, will the spider start grabbing more pages.

Indexing, I think, is an iterative process. From what I've personally seen, which isn't a whole lot, I'm guessing that Google's development team follows a very old Unix-style of programming. Rather than monolithic programs, each part of the system does one task and does it very, very efficiently. The output of one step becomes the input of the next step, much in the way one can pipe data between programs in Unix. I suspect Google is using a FIFO queue, one that DOES get largely (but not fully) cleared on a monthly cycle (as opposed to a continuous cycle, where the queue would always be roughly the same size). If all of this is true, we're not just looking at one schedule, but a series of interdependent schedules. Fuzzy, indeed.

In my opinion, Google's new danceless dance (or continuous dance, if you prefer) is too new to make good predictions. Fifteen days? Thirty? Maybe. But I wouldn't be terribly surprised to discover some new sites aren't included for more like forty-five to sixty days.

[compar]

What is the technical definition of a 'back link'? A web site linking back to you, a reciprocal listing?

Cheers,

James

Yes a backlink is a link to you from another website. You might also call it an incoming link. It does not have to be part of a reciprocal link arrangement although I admit that the term "back" makes it sound this way.

The point is that Google crawls the web by following all the links they find when they revisit pages they already have in their database. If they don't find a link in any of these pages that points back to you they will never find or index your page.

[Jill]

Question:
If I want to work on my SEO, which engine should I go for knowing it would be the faster turnaround.

Back to the original question...

Your best bet would be to pay for inclusion with inktomi, and then you will be automatically respidered every 48 hours. That should be a good way to work on your SEO and get it perfected, before Google comes a'crawlin'.

Jill

[compar]

Back to the original question...

Your best bet would be to pay for inclusion with inktomi, and then you will be automatically respidered every 48 hours.  That should be a good way to work on your SEO and get it perfected, before Google comes a'crawlin'.

I assume you would only submit your home page to Inktomi. If I read their prices correctly it costs $39.00 per year to submit a single URL (the home page) and an extra $25.00 per year for each additional page or URL.

The fact that they talk about URLs rather than pages is slightly confusing. I think a lot of people think of an URL as a Domain, whereas I believe Inktomi calls www.domain.com/index.html one URL, and www.domain.com/contactus.html as another URL.

Have I got this correct?

[Jill]

Yes, it's a per page price.

[market seeker]

BTW this has been said a million times but I still see people who don't seem to understand it, "The only way Google will find a new site is via its backlinks". If you don't have any backlinks you won't get found.

Not completely true
I have submitted a home page to google a couple of different time which had zero links. They were indexed in less than 30 days

[Cruise Planner]

BTW this has been said a million times but I still see people who don't seem to understand it, "The only way Google will find a new site is via its backlinks". If you don't have any backlinks you won't get found.

I hope this doesn't seem like a silly question, but as a newbie, I find the above comment confusing. What's the point of going to Google and submitting...doesn't the submission tell Google that the site is there? Why do I have to wait for Google to find my site if I have already told them where it is?