9 replies to this topic

[Mooro]

I've run Xenu broken link finder on my site today to see what issues it encountered while crawling it.

I also, being nosey, ran it on the site which ranks at one on the mother load / money term to get an idea as to how they get crawled too.

One of the things I noticed was the number of links on some of their pages, their A through Z pages have NO pagination and up to 2,000 links per page, however they each have the highest number of in bound links.

Yes, these huge pages plus the ones they link to are in the index and yes they are ranking well. Very well infact.

I think some site wide A to Z links need setting on my site and I'll up the 150 items per page limit a little......

Guidelines, huh. I'm guessing if you can throw enough on site weight towards a page you can push the limits and get more even more links just a click or two away from the homepage.

As long as there's a good degree of usability for humans on the pages still why not?

[chrishirst]

http://www.webmaster...tml#post1051445

To save a copy and paste

[Mooro]

Cool, thanks Chris.

I've had a good read around and decided to go for it on my A to Z pages by listing all the entries for that letter on the page. 12,000 links in total.

I've already got well placed links to those pages on the index but will add something a little more site wide.

There might well be a lot of links on these pages but my understanding of the supposed limit was to keep pages under 150k

The page with the most links on the first view is 41k then on repeat views of 14k. It's small enough in size.

Having been inspired (sometimes a dangerous thing!) by Xenu I've run a few more reports on different sites to measure the discoverablity of content, in other words how many pages can be discovered by a spider crawling from the homepage down in two minutes.

That's left me wondering about how to increase the number of pages accessible within four clicks of my index....

[Michael Martinez]

It is very rude to run an invasive tool like Xenu link sleuth on someone else's site without their knowledge or permission.

[Mooro]

So what?

--

Dear competitor and online arch enemy,

Please can I crawl 250,000 pages of your site to see how you've managed to get a really deep index and some proper rocking results.

Thanks

Me
PS - any chance of a link please?
PPS - go on gimmi a link.....

--

What would you say to that?

I could predict the second word would be 'off', therefore I just had a little sniff around, I didn't do any harm or anything I consider nefarious, I have the skills to be 'very evil' but never would.

I view it as more of a crime to not conduct a thorough fact find on the enemy than it is to have a little spider of their site.

[Michael Martinez]

The less time you spend fussing over someone else's site, the more likely you won't try to follow in someone else's footsteps. There is no valid justification for this kind of invasive "intelligence". It does more harm than good on both sides of the equation.

You're looking at the shoes that the runner ahead of you is wearing, rather than figuring out how you're going to get past him.

[Mooro]

I repeat: I view it as more of a crime to not conduct a thorough fact find on the enemy than it is to have a little spider of their site.

[Michael Martinez]

I repeat: I view it as more of a crime to not conduct a thorough fact find on the enemy than it is to have a little spider of their site.

I repeat: That is a complete waste of your time as you're not learning anything useful.

[Mooro]

BS.

I got value out of it.

End of, I'm no longer interested in a word your keyboard spews.

[Michael Martinez]

The whole "100 links per page" issue has been explained ad nauseum in numerous forums and blogs.

Like I said: you completely wasted your time.