45 replies to this topic

[TimS]

Welcome TimS!

Jill

Hi Jill I hope I wasn't too specific when I addressed my post to Grumpus

Feel free to jump in any time, I'm trying to figure out what the best way to handle all those pages would be

One more note, as a fellow active forum owner, you set up a wonderful place here

TimS

[Grumpus]

Seems to me that you could whip up (or find with a good Google search) a php script that'll read the directory structure and spit out the first say 50 results, then paginate to the next 50 results and so on. This wouldn't be ideal for topical structuring for search engines, but it'd be better than nothing.

The ideal situation would be to hardlink from the current forums to the specific relevant posts. In other words, make a new post that says, "Here are the best posts from our old board dealing with such-and-so" and then link to 'em.

G.

[Tawnya]

You have:
Me -> About Me
Me -> Portfolio
Me -> Services
-Me -> Services -> Graphics Design
-Me -> Services -> Web Design
-Me -> Services -> Seo

I have been reconsidering my hierarchy of my website towards an example like you have stated, with each service having its own directory for the past few months yet have not risked doing it yet since I'm pulling #1 in most engines for my keywords "virtual assistant canada". I know this will disrupt my listings in the SERPs having new directory names/files, etc so have yet to do it. I would however like to "theme" my site more (and also make it more manageable/organized for myself) and at some point feel it is inevitable. Should I slowly move the pages or just dive in and do it all at once? What do you recommend?

As for the sitemap, I'm willing to remove mine from my site for a test run, see what happens. Reason I've always had it there was for usability purposes to help the user if they got lost.

Tawnya

[Grumpus]

I'd do it over time. Move one section (and make sure you put a 301 redirect placeholder in place of the old page in case there are links going to it and to cover yourself when your old pages are still in SE indexes).

I never like to make any radical changes to a site. Ideally, it needs to be set up so that that's the way it'll always be structured, but as you know, that's not always the case. So, gentle nudges are going to be the way to go. Hopefully you'll have access to your .htaccess file so you can properly redirect everything. Learn how to do that first, and then try it with a few pages.

G.

[rustybrick]

I just had to say, this is one of the best threads I have read in a really long time.

G. Thanks so much!

[Grumpus]

Thanks Barry. It has been a fun topic for a while now.

G.

[TimS]

The ideal situation would be to hardlink from the current forums to the specific relevant posts. In other words, make a new post that says, "Here are the best posts from our old board dealing with such-and-so" and then link to 'em.

G.

Thanks Grumpus Maybe it's just me, but after seeing your picture next to your posts, everytime I read them I swear it sounds like Stephen Baldwin talking Sorry...it's late...I really don't know you folks well enough to tease yah yet

As to linking to those threads from the current forums, I had considered that. Two things stand in my way:

1. I'd have to re-read about 75,000 threads and categorize them.

2. The primary reason for wanting to somehow index them and "feed" them to the spiders a little at a time was to bring them a few PR levels closer to the main page of the site. One thing I've noticed, at least on our site, is that each click away from the main page is a drop of 1 PR point. Dunno if that's normal, but that's how it is currently. The forums, being a different script name and all, don't even have a PR rank yet - and they are only one click from the PR5 main page. I imagine the forum index will probably be PR4 - and then individual forums PR3. If I can index these old pages and feed them to the spiders from the main page, I'll bypass the forum index and suspect that I'll only lose 1 PR point instead of the suspected 2 if I put those indexed pages in the current forums as posts.

Or am I completely off my rocker? Which is entirely possible at this point

TimS

[Grumpus]

Don't worry about teasin'. I've been called a heck of a lot worse than a Baldwin. Many of them unable to get around the forum filter.

The PR going down one per click is fairly normal. It naturally happens because of the nature of PR. The deeper the page, the less likely there is to be a direct link from an outside site, so you're relying on your own internal linking structure to pass PR. And, try as we might, we can never pass more PR than we have.

Another factor is that it takes a PR update for PR to be calculated. Often a page is found, makes it into the index during one update (yes, it's a little different nowadays, but...) and a "guessed" PR will appear for another month before real PR gets applied to the page. It takes an extra month for real PR to be calculted for the page. On dynamic sites, I've seen it take as long as 6 months for a page to get real PR - mainly because it can sometimes take Google that long to identify what it is that makes the page a unique call, I suppose.

At the end of the day, though, it reaches a point where you just need to kinda start treating PR for what it is and not take it too seriously. You need a good base PR and you need good inbound links. It's something you should always be working on, but when it comes down to it, PR doesn't do a heck of a lot for ranking pages - espcially deep ones. At a certain point for "specific terms" (which your deeper pages should be) you can blow a PR6 or 7 page out of the ocean with a PR3 page. This is where your link structure and on-page elements come in.

The trick here is that we don't know what's on each of the various archive pages you've got until we look at them. So, it'll be pretty hard to telegraph anything. The pagination of the various directories as I suggested in the first post may work, but you're not going to get any useful inbound link text in there. I can't (with the limited knowledge I have on your site) imagine a programatic way of handling this optimally. You can whip something up and it'll get in the index and call it a day. Or you can hand link to things optimally but you'll be at it for a decade and by then, the linking rules will have changed.

If your site is fairly active, maybe you can generate some interest in getting others to help you out. Send the teeming millions in there and get them to start linking to their favorite ones. If it's like most forums, maybe 10-15% of the posts are really worth indexing for posterity. Discussions and debates are fun while they are still active, but have little value once they'd died out. The constant repetition of the same old questions (such as here and the "Help! My Site Is Gone!" posts) is natural because it's hard to organize a forum so that it's easy to find things. There's nothing wrong with them, but there's no particular use in reindexing them.

<shrug> It's a tough call because there is no right answer. There are plusses and minuses to each option.

Not an answer, I know, but hopefully it's useful in some way. And now, I'm off to shoot a movie.

-Steve B.

[blackpool]

Grumpus,
The article that you pointed out, about PR, earlier in this thread is brilliant.
Cheers for that

[TimS]

Thanks again Grumpus, for taking the time to put that all into words, I appreciate it.

I guess in the short time I've learned about this PR that I've become overly concerned with it...it just seems cool, I'd love to see 6

One last question on this subject...cause I gotta about 3 more question on other subjects I'd love to hear about from the folks here

Is there any inherently positive thing about having an enormous amount of pages, say like 50,000...even 100,000 pages, in any search engine index? Besides the fact that internally, they'd all each give a little tiny piece of PR back to the index page of the website

I'll try to find the appropriate forum here for my next question....it's a Googlebot question...I was handfeeding one yesterday morning and....

TimS

[Grumpus]

Aside from PR, a lot of pages is the difference between shooting a .22 rifle and a big old honkin' shotgun full of birdshot with a nice wide spread. With the rifle, that one bullet's gotta hit home. With the shotgun, one of the pellets is bound to hit.

I wouldn't go making a site with millions of pages just for the sake of having millions of pages, though. Each page should have enough unique content to make it worth something.

G.

[TimS]

Thanks G, I'll get the crew to sift through those pages as you suggested earlier.

TimS

[Ruud]

-- note -- due to a small accident i can only type lower case mostly, sorry


for the searches http://tinyurl.com/25szp = micmac culture and http://tinyurl.com/yrxd5 = micmac history my basket page now comes up as more relevant than the actual pages that are about these topics. http://tinyurl.com/3c2ef = micmac community still leads to the relevant article. only difference i see is the h tag there.

this is great: google takes my index page of micmac articles to be more relevant than the articles itself. it seems to know the index page has more info about micmac than the articles itself - or the keyword content ratio simply is higher this way.

ruud

[Grumpus]

Sure. Because of the things I outlined above, your page about pages that are about the topic are more relevant to the general topic than a specific article. That one page takes you to all the pages. Google has recognized your structure and now your index page is acting as a mini hub/authority page within your site.

It's nice to hear when techniques are applied and they start to work. Love it!

G.

[Ruud]

yes, that is why i thought i would mention it. it is a nice theory - and one that definitely works.

ruud