5 replies to this topic

[Ruud]

My host uses Zend so obviously there is some on-server caching provided already. Now I wonder if it is useful to use an additional caching mechanism, say Smarty templates for example?

Ruud

[Ron Carnell]

I've never tried it, but doubt it would help and suspect it might actually hurt.

The idea behind caching is that it's quicker to find something in memory than to find it on disk, but there is always a point of diminishing returns. That's because caching adds processing overhead, so the savings have to be greater than the overhead to reap the benefits. Usually it is, but there are two common instances when it isn't. If the cache is too large, the overhead of managing it can be greater than the savings. And, more pertinent to your question, if the caching is parallel you'll typically find yourself adding overhead without ANY appreciable savings.

Think of it this way. "If I need X, I'll look in Cache A for it. If I don't have it in Cache A, I'll then look in Cache B." Trouble is, unless the two caching algorithms are substantially different, they'll usually end up caching the same stuff. You either never get to B because it was already in A (in which case, the overhead of putting things in B is wasted), or you waste the time of looking for it in B when it's almost never there.

[Ruud]

Hm yeah, I see what you mean. I however would cache on disk. Write out normal files and serve them instead. I don't see much use in using a template engine with caching ability (such as Smarty).

PHP and MySQL already do a lot of caching themselves but with *very* intense operations I still can get a max_questions error/warning (MySQL) on my host. I have an on-disk caching system (static files) built in already just in case my site ever would become so popular normal visitors can cause this to happen.

Ruud

[bobsledbob]

Boy, you're really not helping us out very much here. There are so many levels of optimization and caching that you can do, that without knowing more specifically about your situation, it would be hard to offer advice.

However, I've got some experience building some high traffic highly dynamic sites, and so here's some of my experience...

o The biggest bottleneck will (should) always be the database. MySQL is fast for reads, but doesn't in general have some key functionality that could make it faster (stored procedures, sub-queries, etc.) in real world use. So, what usually happens with PHP and MySQL applications is that you perform many individual queries from PHP to MySQL. That is, you fetch a set of results, then you dump it into a PHP array, and then you loop through the array and fetch more results for every entry. This is pretty typical bad architecture that will lead you into trouble.

The point is, if you can't switch databases, you really need to examine your code closely to avoid looping application queries, etc. that would cause problems. Learn (or hire someone) to do quality database queries, with joins to avoid looping structures.

o Also, I've found that for websites, there's very few times when a live call to the database is really needed. Particularly, if you're dealing with static data or mostly static data, you should really be able to avoid calling to the database on every page/script/etc. While there are all sorts of useful caching mechanisms, I've found the best ones are the ones which I've written myself.

For instance, say you're populating an html form select element with a series of data. This data resides in the database, but doesn't really change from page to page. So, you build a plain text cache of the data. When you update the data, you refresh the cache. Could be as simple as a comma or tab delimited text file or something.

With Linux and other decent operating systems, frequently asked for files will be moved into an in-memory file cache. Once it's there, you'll get great performance. I've for instance got a 1MB text file with about 4000 rows x 50 columns of tab delimited data that I read with a PHP script. The Linux kernel caches this and gives it to the PHP application very quickly. I have in the past got maybe 10,000 hits per day to this script, no signs of even slowing down. I think my load was at .05 or something (very low). This is just an example though of thinking about your data and how to reduce database queries.

o Using template engines like smarty only makes sense when you're wanting the template functionality that goes with it. If you're not familiar with template derived designs, what essentially happens is that your data gets seperated from you business logic which gets seperated from your display. This is more formally called 'Model 2 Architecture' and is a great coding paradigm for large sites.

However, to use a tool like smarty just for its caching would be a bit silly. It has enough overhead that using it for caching only wouldn't be justified.

o If you still want some caching solutions, then do some more searching or visit pear.php.net There are a couple of decent caching implementions in there that can be very good for various situations.

o Also, don't forget to be checking out some web server improvements. By default, any PHP script will send a no-cache header back to the web browser. This means that even though the page has likely not changed, the browser must request the whole content again.

Instead, you should be sending headers that instruct the web browser to store a local cache of the data. You could do this in PHP or you could do this with your web server. Either way, unless you're managing say a user's shopping cart or something, you can pretty much guarentee that browser can and should cache the content.

In my example above (with the 4000 row text file), the application pages through this text file showing maybe 30-40 rows per page. Thus, the user has to click 'next' to get to the next page. But, if the user clicks the browser's back button or if the user clicks the 'previous' link, the page will be in the browsers cache and the user moves there instantlly without even a new http request.

o When you say 'running Zend', this doesn't tell us too much. There are two probable products which you're referring to.

One, the Zend Optimizer is a product that will run encoded PHP applications and also do some code optimization tricks that can yield decent improvements. This is a free product.

Two, the Zend Accelerator, is a caching engine that hooks in with the PHP engine. It essentially caches the bytecode of your scripts and then executes these instead of recompiling the PHP script each time its requested as is normal. Very decent gains can be had with the Zend Accelerator, but this is not a free product.

Usually most hosting providers wouldn't have this installed because it's not free. However, there are a couple of very good free accelerators out there. Yahoo uses a product called ionCube PHP Accelerator http://www.php-accelerator.co.uk/ for their PHP implementation. It's a very good product.


Ok, to summarize, check your code. If you're running into bottlenecks, and you're not serving content on the scale that Yahoo would, then you've likely got application related issues that could be addressed. Most small to medium sized websites will perform just fine on a standard web hosting account. It's all a matter of making sure your architecture is the best it can be. If you don't know how to do this, there are lots of hungry developers who would likely give you a hand.

- Adam


Oh, and a note to Rob Carnell, while your example is fine, the zend optimizer and zend cache both run at quite a different level than does smarty or other PHP centric cache solutions. So, in actuality, the two types of caches work very well together. The two types of caches are vastly different.

[bobsledbob]

Hm yeah, I see what you mean. I however would cache on disk. Write out normal files and serve them instead. I don't see much use in using a template engine with caching ability (such as Smarty).

PHP and MySQL already do a lot of caching themselves but with *very* intense operations I still can get a max_questions error/warning (MySQL) on my host. I have an on-disk caching system (static files) built in already just in case my site ever would become so popular normal visitors can cause this to happen.

Ruud

I just more carefully read this reply. I think I provided some redundant information in my previous note. Sorry.

If you're writing out static html files to be served directly from the web server, you're taking a big step. So, nice job.

I'm pretty sure that you just need to carefully examine your database model and your queries. Like I said, because MySQL doesn't have stored procedures and sub-queries, it's very easy to write applications that abuse the database, such as what you're describing. It really sounds like you're doing exactly what I said, that is querying the database for a result set, then iterating on that result set with more queries.

Much better to have a good SQL join statement that can remove this necessity. In the ideal world, you could make one query and get back all the data you needed.

Even more ideal would be to switch databases for something better. PostgreSQL comes to mind. You get stored procs and subqueries with it. All sorts of other goodies too.

[Ron Carnell]

Excellent post, Adam. And yes, of course, if caching operates on different algorithm levels, they can certainly coexist well. I think I implied that? Since I don't know anything at all about Smarty or its function, my response was pretty generic.

I particularly appreciated your point about optimizing code as a first-step measure, and would add that also applies to the underlying database design. Poorly performing code or badly normalized data are at the heart of 99 percent of all load problems. Today's iron can just about perform miracles … with just a little bit of help.