34 replies to this topic

[vtownbarries]

We have different categories which is dynamically served from a database.
Now we need to get all that information indexed by google ... but it only indexes the getcategory.php?xxxxxxxxx what ever the first parameters are.

Is there a way we can write the site so that it will go through the whole site ...
but not to statically write the site?

[linux_lover]

Yeah you basically use php to use the url as a variable employing apaches lookback feature

so domain.com?item=flower&id=56 becomes

domain.com/flower/56

Theres lots of good tutorials do a google search on 'search engine friendly urls'

[vtownbarries]

Cool thats exactly what I need ...

The way you use the 404 error page is not ideal for me as I need the error logs to check if anything happens to the clients info etc ...

Cheers

[strangerrr]

Hi

It is fact that most search engines cannot or will not list any dynamic URLS. Dynamic URLs are said to contain elements such as ?, &, %, +, =, $, cgi-bin, .cgi. Hence, the pages which the hyperlinks point to will be ignored by the Web spider indexing the site.

It is very easy to create seacrh engine friendly URL using apache. In apache we can use MOD REWRITE Function in the .htaccess file.

[Grumpus]

Erm. No, it's not a fact.

Here's just a sampling of pages that use querystring variables. Yahoo and MSN (the only other real search engines out there that have spiders) are both equally capable of crawling dynamic URLs also.

G.

[ramorse]

I have a similar question. I have been hired to work on a site that has product pages in a shopping cart listed like http://domain.com/ca...etail.cfm?id=51

None of those pages are listed in Google. But they are in Yahoo.

None of those pages have title tags, either so that may be an issue.

I was told in another forum that if you end your url with id=xx Google won't index it because Google won't follow links with query strings that may contain User ID numbers or session ID
numbers.

But I have found other pages in Google witn id=xx endings.

[BrianR]

Hello and welcome to the forum, vtownbarries.

BrianR

[vtownbarries]

Thanks for the welcome ...

So what you are telling me that is i structure the site to be as the following :


www.domain.co.za/getcategory.php/pens/paper/etc

will apache not look for that folders in your www folders ...

I found this site that show you what functions to use in php to decode the url but I am a bit afraid a url like that wil not go to the right php file

www.sitepoint.com/article/search-engine-friendly-urls

Anybody have any experience or advice regarding this?

Thanx

Edited by BobetteKyle, 26 November 2004 - 11:53 AM.

[Randy]

ramorse: The "id" thing is a myth these days. It's not an issue with any search engine anymore as long as the ID is a product or category ID and not a Session ID.

[Randy]

vtownbarries: What you're talking about is using mod_rewrite to change your URLs. It may or may not be needed. Yes it can help if you have multiple modifiers in your URL --for example page.php?var1=some&var2=else&var3=entirely -- but if you don't have multiple variables/modifiers it's likely not going to solve anything.

I stuck a bit of mod_rewrite code in this thread but it can get fairly complicated. You have to be careful and each rewrite situation has to be looked at individually in my experience. I would caution to look for other issues/solutions first.

First check to make sure the site doesn't require a Cookie to be set. It's pretty common with many commercial apps and stops the bots in their tracks. Also check to make sure a Session ID isn't being set.

If you want me to take a quick look at the site to see if I see any problems, PM me the site's URL. I'm more than a bit swamped today, but I promise to take a look as soon as possible.

FYI, more links help to get dynamic sites spidered. They tend to be larger by their nature. More backlinks = more interest. So the spiders will crawl deeper. The fix often is as simple as that.

[ramorse]

Randy: That's what I am learning. That the id=xx is not the issue. So then my problem becomes why are the pages not being indexed?

[vtownbarries]

My first success in getting the pages indexed is to use dynamic <title> tags ...
And also include meta tags where your keywords are dynamic as well.

You guys can have a look at www.easyinfo.co.za.

My question is if you go down a category ... say plumbers ... the list is not in the same order every time (we use random numbers).

Now my question is does spiders crawl the Next button on each page?

I cant find any other answers so your help will be largely appreciated.

Thanx

[simplerchoice2004]

Well, in theory there's no reason why the next pages wouldn't normally be picked up as it's simply another link on the page for a spider to crawl.

However, my understanding is that your use of frames (which Google never used to like very much but is moer tolerant of now) and multi-parameter query strings (6 parameters in places) will not help you get listed in Google.

BTW, I also noticed that your title for each page is not being generated correctly. Instead it's being shown as :

<title>easyinfo <? echo "-" .$searchword." ".$desc_cat; ?></title>

I suspect that this is due to the frameset page being a .htm rather than a .php and therefore it's not being parsed as PHP.

From a usability point of view, I'd recommend trying to keep plumber (and other category) listings in the same order. There's nothing quite as annoying as going to a page of plumbers, looking at the top result and then going back to the same page later and finding that result isn't there anymore.

Kind Regards,
Simon

[vtownbarries]

thanx for the title warning ... i fixed that...

i also find that many people encourage the use of
<META name="ROBOTS" content="INDEX,FOLLOW">

any opinions on this ...

then I would like to ask if there is a good or easy way to test how your rankings are at the moment ... what methodology to use?

Thanx

[chrishirst]

Robots Metas