13 replies to this topic

[Brutus]

Hi,

I've searched for an answer to this but not been able to find one yet.

I have a page which lists links to all of our members profile pages and there are about 1700 links all on this one page. I need this as otherwise the SE spiders cannot find these pages and will not index the content (sorry if my terminology is dodgy, but I hope you get what I'm trying to say).

I would have expected Google to index all of these profile pages by now but it only sees around one hundred of them. It would be really helpful if I could get these pages indexed as there is quite a lot of content within them that should bring me some more organic traffic.

The page in question is: (Removed. See [url=http://www.highrankings.com/forum/index.php?act=boardrules]Forum Rules[/url].)

My developers are not SEO experts by any means so I would be grateful for any help.

Thanks,

Dave

[Randy]

A general guidance the search engines have given over the years to limit the number of links per page to 100 or fewer.

Now you know why.

You'll need to break things up into multiple pages, or provide the spiders a different way to get to those pages.

[Jill]

There's probably some way you can categorize the members, perhaps by state?

[Brutus]

The site uses Ajax and it would be fairly simple for my developers to seperate the members A-Z by surname so that clicking on a letter would list just those members. Would this actually help though because all the links would still be located within a single URL?

Thanks for your advice. Without this forum I would be screaming at the walls!

[torka]

I think they'll have to be on separate URLs. The SEs don't care about the visual presentation, so just breaking it up visually (which it sounds like what the AJAX thingie will do) won't make any difference to them.

They care about the content of the page. One URL = one page as far as they're concerned. You need to get it down to 100 or fewer links per URL/page.

--Torka

[Randy]

Have your developers bone up on Progressive Enhancement. (That's at least three times I can think of where it's been the solution in the last week or so!)

Done right, it'll allow the spiders to get to all of them because they'll get the same url with some different variables, but the user experience won't suffer or cause a complete page reload for each choice.

[Brutus]

I've mentioned Progressive Enhancement to my developers but I don't think they understand it in terms of how it can help in this case. I will try and make some sense of this but if I can't get anywhere it's looking as though I'll have to split the list into lots of different pages. Not ideal for usability but I do need them indexed to help bring in traffic.

Regarding the 100 links advice.... Each page has about 40 links just with the menu icons and similar links. Does this mean I have to try and limit additional links to around 50 or will the spiders understand that the other links are necessary navigational links and not count them as part of the 100 limit? (I know 100 is probably a guideline figure so I use it loosely here).

[BBCoach]

I don't know where Randy got the 100 number (I've never seen that recommendation). Also, it doesn't make sense since they want sitemap pages for all of a web site's links. One of my sites has more than 1000 internal links on one page to internal pages and those pages are crawled daily by Google, Yahoo and MSN SEs. That page is categorized by state though and there's also a sitemap page. Perhaps that's the difference. That primary page is indexed and ranked in all of the SEs. Sometimes instead of the page with all of the content. Go figure.

[simona]

is ajax not a form of javascript?

[Jill]

I don't know where Randy got the 100 number (I've never seen that recommendation).

For years it was the recommended number on Google's tips for webmasters. I rarely read that, so I don't know if they're still saying that or not.

One of my sites has more than 1000 internal links on one page to internal pages and those pages are crawled daily by Google, Yahoo and MSN SEs.

Yes, just because there's one page with 1000 links, doesn't mean those 1000 pages won't be crawled. It could very well be that they were found through different means. Or it could mean that Google crawled all 1000 from that one page. One would hope that there were other ways of reaching those pages besides that one page, however, because if not, they're not going to be given much PageRank (the real kind not toolbar) and therefore won't have much chance of showing up in any search queries.

[Randy]

It's still Google's recommendation. (See the section on Design and Content Guidelines. The 100 number is actually menitoned twice now.

[Brutus]

These links point to pages that are normally found via a user searching the directory. I understood that spiders cannot follow these type of directory queries and this is why I asked for a list to be generated. Perhaps it is worth splitting up this list into a page for each leter of the alphabet.

Although the site does have around 150 inlinks I have yet to start a proper link campaign targetting some larger more authoratative sites. Maybe Google doesn't rate the importance of my site highly enough yet to crawl it properly?

[Randy]

Maybe Google doesn't rate the importance of my site highly enough yet to crawl it properly?

Quite possible.

As a general rule, the more important they consider your site, the deeper they'll crawl.

[BBCoach]

Wow! I bet I've read that page 100 times, but not recently until now. Learn something new every day. Prolly knew it and just needed a refresher being I'm an old geiser. Thanks Randy. Also, I think that there must be outdated info on that page. Notice how G also says (just above the last mention of 100), "If you decide to use dynamic pages (i.e., the URL contains a "?" character), be aware that not every search engine spider crawls dynamic pages as well as static pages. It helps to keep the parameters short and the number of them few." We know today that isn't an accurate statement (at least not for the top 3 or 4 SEs), and for those that don't crawl 'em we don't care about.

Jill, as with Brutus' site, the site I'm talking about has a form post to get to the internal pages. That one page and a sitemap page are the only way to get to all of the internal pages without posting a form. Those internal pages when crawled and indexed are ranked in the top 20. Usually in the top 5. In all three SEs. I don't know what else to say, but that's the results I get doing exactly what Brutus is doing. Maybe it's an exception to these recommendations and Brutus would be wise to heed those.