Benchmarking PHP accelerators

How does an accelerator works ?

Executing a PHP scripts takes a few steps :

1. PHP loads the file,
2. it parses the source file, and transforms it into opcodes (code that can be executed by the server),
3. it executes the opcodes.

The accelerator takes the opcodes from step 2 and caches them in shared memory or on disk. Those cached opcodes are then directly reused the next time the PHP file is executed, without loading & parsing the file again.

Some accelerators add an optimization step which removes unnecessary code (empty loops, unused variables, …). In most cases, this optimisation step does not improve performance much.

Available accelerators

There are 3 accelerators (at last 3 that are stable, maintained and fast) :

• APC (open-source) - http://pecl.php.net/package/APC,
• eAccelerator, a Turck-MMCache fork (open-source) - http://eaccelerator.net/,
• Zend Platform (commercial) - http://www.zend.com/products/zend_platform.

A fourth one was released not very long ago : xcache (http://trac.lighttpd.net/xcache/), from the developers of lighttpd. xcache and Zend Platform will be tested soon, and included in this benchmark.

Only eAccelerator and Zend really try to optimise opcodes.

Zend Platform is much more than an opcode cache : it includes a couple of tools (debugging, monitoring, Java integration) that can be of interest. Discussing those feature is not the subject of this article…

First round : support and maintenance

Zend Platform has roughly the same performance as eAccelerator and apc - I’ll include the figures as soon as I’ll be able to finish the tests, but is the only one to have an official support. If you need support, or the specific features of Zend Plaform, and are able to pay for the software, you’ll probably end using Zend.

eAccelerator once was a dead project, but the current maintainers do a fine job updating the code. The current version is labelled beta. It can be frightening, but the whole notion of beta is very relative (some would say that well-known companies launch beta-quality products labelled as version 1.0). I know quite a few people using this beta version of eAccelerator on production servers without glitches…

apc is being developed by a team of core PHP developers, including Rasmus Lerdorf, the inventor of PHP. apc is planned to be included in PHP6. It is difficult to have a better future… Yet eAccelerator has a larger user community, and it is often easier to find information or help on eAccelerator than on apc.

Second round : performance

We will now compare the performance of apc and eAccelerator…

Test platform

Tests are done using two Linux servers (one client running the stress tool, the other one housing apache and PHP) :

• Xeon 2,8GHz (without Hyper-Threading),
• 1Go RAM,
• Apache 2.0,
• PHP 5.1.4 (last version available on 5/17/2006).

The versions tested are the last downloadable ones (cvs versions are not tested - most people would not check cvs).

The eAccelerator version used is 0.9.5-beta5 (previous versions do not support PHP 5.1). eAccelerator is compiled with default options and configured with :
eaccelerator.debug=0
eaccelerator.shm_only=1
The APC version used is 3.0.10, compiled and setup with all the default options.

Methodology

3 test scripts are used :

The first script (hw.php) is our good old “hello world”.
<?php echo "Hello world"; ?>
The second script (deserialize.php) loads and deserializes a 92KB object.
<?php
$filename = "obj/serialized.obj";
$str = file_get_contents($filename);
$obj = unserialize($str);
if (is_object($obj))
echo "OK";
else
echo "NOK";
?>
The third script (include-pma.php) is based on phpMyAdmin (version 2.8.0.4). It loads the main include (for a total of 14 included PHP files). PMA_MINIMUM_COMMON is set, so that it does not try to connect to a database. Because the script uses disk-based sessions, the sessions directory is emptied before launching the test.
<?php
ini_set("include_path", ".");
define('PMA_MINIMUM_COMMON', 1);
require("./libraries/common.lib.php");
?>
Load is generated with apache bench (ab2), configured with 20 simultaneous threads (-c 20), and a total of 5000 HTTP requests (-n 5000).

A warm-up test is first lauched in order to prepare apache for the load, and a single request to the test script is made in order to populate the cache with all PHP files.

Each test is iterated 3 times.
ab2 -n 5000 -c 20 http://xxx/xxx.php

Results

Here are the results :

Compared performance between PHP/apc/eAccelerator

(unit : HTTP requests/s)

The more complex the code, the higher the gain. A script using lots of large includes will be much faster thanks to the accelerator, but a slow one (ie : deserialize.php) won’t.

If you have lots of database or disk accesses, an accelerator won’t solve your performance problems.

Both accelerators have similar performances, we declare this second round a tie.

Third round : file updates

We will know simulate code updates, while running the include-pma.php test. This will enable us to measure how both accelerators handle updates to their opcode cache.

First test : in the middle of the script, one file is updated
touch libraries/common.lib.php
Second test : in the middle of the script, all files in the libraries directory are updated (9 files are reloaded)
touch libraries/*
Here are the results :

Impact of updates on performance

(unit : HTTP requests/s)

Quite a difference between both… eAccelerator is obviously much better at updating its cache. If you do modify your code when you have high traffic, apc might cause you some trouble.

In some cases (very high load and lots of code updates), apc might not even be able to converge (being unable to cache all the modify files). The apc.slam_defense is meant to ease this convergence. But, whatever the value of apc.slam_defense, apc won’t be as fast as eAccelerator.

The apc guys are currently working on this problem. The current cvs version of apc adds a new “write lock”, that is much more effective than the previous locking mecanism. Preliminary tests show no degratation of performance when updating files with this new version. We’ll wait for the official version before adding the figures in the benchmark.

Yet, one should point the fact that apc is better than eAccelerator at maintaining the integrity of cached code during slow file transfers (eg : FTP transfers). apc waits a few seconds to be sure that the transfer is finished, whereas eAccelerator tries to execute partially transfered code (usually resulting in a handful of Fatal Errors during the transfer).

Large advantage for eAccelerator for this last round

Conclusion

The tests we did showed us there can be an important performance gain by using a PHP accelerator. Using a PHP accelerator seems essential for a high-traffic site.

But PHP accelerators are no panacea : if your code is slow, it will stay slow even with apc, eAccelerator or Zend.

Choosing an accelerator depends on your criteria :

• if you can pay for Zend, and need an official support, use Zend Platform,
• if you don’t, eAccelerator is a good choice.

Each tool has advantages and drawbacks, the only way to know which one is the best at handling your code is to test both.

If, as I did, you choose eAccelerator (in order not to spend a dime on it, and to be able to change your files whenever you want), I would advise you not to get too much tied to eAccelerator, you might have to switch to apc in the future…