Some performance hacks for zp #2

invarbrass

See related thread: http://www.zenphoto.org/support/topic.php?id=3172&replies=1

#2 Bypassing directory traversal
ZP scans the albums directory every time the script is executed. Since disk operations are expensive, we can try to limit disk-based I/O to make it load faster. We'll be using eAccelerator to cache our stuff, but you can easily adopt this code for APC, XCache, memcached or even a disk-file.

* Open class-album.php in zp-core directory.
* Put the following code at the beginning of the script:
`

function store_filenames_cache($dir, $has_dir, $pic_cache, $vid_cache) {

$cache_time_out = 600;

if ($has_dir) {

$pic_key = 'dp_' . md5($dir);

$vid_key = 'dv_' . md5($dir);

}

else {

$pic_key = 'fp_' . md5($dir);

$vid_key = 'fv_' . md5($dir);

}

$pic_val = serialize($pic_cache);

$vid_val = serialize($vid_cache);

eaccelerator_put($pic_key, $pic_val, $cache_time_out);

eaccelerator_put($vid_key, $vid_val, $cache_time_out);

}

function get_filenames_cache($dir, $has_dir, &$pic_cache, &$vid_cache) {

if ($has_dir) {

$pic_key = 'dp_' . md5($dir);

$vid_key = 'dv_' . md5($dir);

}

else {

$pic_key = 'fp_' . md5($dir);

$vid_key = 'fv_' . md5($dir);

}

$pic_cache = eaccelerator_get($pic_key);

$vid_cache = eaccelerator_get($vid_key);

if ( $pic_cache != NULL ) {

$pic_cache = unserialize($pic_cache);

}

else {

return FALSE;

}

if ( $vid_cache != NULL ) {

$vid_cache = unserialize($vid_cache);

}

else {

return FALSE;

}

return TRUE;

}

`
* Locate the loadFileNames() function, and replace it with this:
`

function loadFileNames($dirs=false) {

if ($this->isDynamic()) { // there are no 'real' files

return array();

}

$albumdir = getAlbumFolder() . $this->name . "/";

if (!is_dir($albumdir) || !is_readable($albumdir)) {

$msg = gettext("Error: The 'albums' directory")." (" . $this->albumdir . ") ";

if (!is_dir($this->albumdir)) {

$msg .= gettext("cannot be found.");

} else {

$msg .= gettext("is not readable.");

}

die($msg);

}

//masroore - cache these results

if (get_filenames_cache($albumdir, $dirs, $files, $videos) === FALSE) {

$dir = opendir($albumdir);

$files = array();

$videos = array();

while (false !== ($file = readdir($dir))) {

if ($dirs && (is_dir($albumdir.$file) && (substr($file, 0, 1) != '.') || hasDyanmicAlbumSuffix($file))) {

$files[] = $file;

} else if (!$dirs && is_file($albumdir.$file)) {

if (is_valid_video($file)) {

$files[] = $file;

$videos[] = $file;

} else if (is_valid_image($file)) {

$files[] = $file;

}

}

}

closedir($dir);

store_filenames_cache($albumdir, $dirs, $files, $videos);

}

if (count($videos) > 0) {

$video_thumbs = array();

foreach($videos as $video) {

$video_root = substr($video, 0, strrpos($video,"."));

foreach($files as $image) {

$image_root = substr($image, 0, strrpos($image,"."));

if ($image_root == $video_root && $image != $video) {

$video_thumbs[] = $image;

}

}

}

$files = array_diff($files, $video_thumbs);

}

return $files;

}

`

invarbrass

Here are some benchmarks (10000 iterations):
original loadAlbums() = ~ 20 secs
optimized loadAlbums() = 0.89 secs

Don't forget to change $cache_time_out according to your preference (default is 10 mins).

acrylian

FYI I have already added it to a new article on our extensions page called "performance hacks" (together with the other hacks we already had in the other "hack" article). It still does not show up because of the wp-cache though.

invarbrass

thanks! but the code formatting is all messed up here. perhaps you could pretty print it on the blog page?

acrylian

We don't really have code formatting on the blog either, we tried a wp plugin, but that messed up, too...:-)
For future "hacks" the best would be anyway if you attach them as files to a ticket.

trisweb

Trac has much better code formatting, maybe we should use it for these hacks pages?

acrylian

Well, then we again have the stuff in several places... Best (and space saving) would probably be to have the actual files for download instead of printing them. The comments could include the usage instructions.

jsumners

I intend to begin working on integrating Smarty into ZP next month. It has a caching system built-in. I'll make sure to integrate the caching as well.

acrylian

Ok, if you wish so, I actually can't remember that we wanted to do that. But if you make that as a "optional hack", that's ok.

jsumners

My intention is to do it as an evaluative branch. trisweb was on board with implementing such a templating system for a 2.0 feature, so that leaves you and sbillard to come around to the idea.

There is really no way to make it optional. It is a total restructuring of the logic and presentation code (that is, it separates the two).

trisweb

There is definitely a way to make it optional... Zenphoto's logic is separated from the presentation already, you'd just be replacing the template-function presentation code with your own to support smarty. The classes are the backing model and they'd stay the same. The controller might need to be changed to support different navigation schemes. Other than that I don't see a problem using both simultaneously.

Zenphoto can support smarty as a feature, but I'd prefer it not to be the only option. Of course I'd be willing to give you a branch to develop in, might as well see where it goes.

jsumners

I intended to get in touch with you about it closer to the time when I can work on it. I'm still wrapping up my senior paper, and I have chemistry lab for the first half of summer semester. So I don't have time to look into it fully right now. It's just something I want to explore when I am able to get back to hacking on ZP. I hold no delusions that if I do a lot of reworking that it will be accepted into the trunk automatically.