Unregistered
As noted in the Zenphoto 1.4.2 changelog, there's a(n unsupported) plugin for pre-caching images here, if you truly find it necessary: http://www.zenphoto.org/index.php?p=news&title=cacheImages
I'm not entirely sure of the reason beyond what's already been said, but I would venture that, at least from my reckoning, it does seem unnecessary. In the case of a large-scale site, the images will be cached presumably quickly as there would be a multitude of users to view the image. As for a small-scale site, it doesn't seem as though it would put any sort of strain on the server to process and serve images on the fly.
I can understand your reasoning for wanting this functionality, though. I'm not entirely sure that pre-caching will yield higher usability, since processing a single image at a time should be quick, but this is exactly why the above plugin was created.
I'm not entirely sure of the reason beyond what's already been said, but I would venture that, at least from my reckoning, it does seem unnecessary. In the case of a large-scale site, the images will be cached presumably quickly as there would be a multitude of users to view the image. As for a small-scale site, it doesn't seem as though it would put any sort of strain on the server to process and serve images on the fly.
I can understand your reasoning for wanting this functionality, though. I'm not entirely sure that pre-caching will yield higher usability, since processing a single image at a time should be quick, but this is exactly why the above plugin was created.
Member
Member
Great software, great support.
I'm going to plus one this old post and reopen an old and ongoing flame to add to the vote, add my reasoning,add a little comparison to gallery3, and some observations about caching performance in zenphoto.
First it's clear there is little reasonable argument that making a browser ever (even just for the first user) wait to produce an image that could have been produced before is a user feature. Placing the fancy words ''on the fly'' on it doesn't change that fact.
This is primarily a coding feature, not a user feature, and the nice sounding words apply in that context. I suspect it makes it far easier to allow themes to use any size image and not worry about it, and probably other similar brilliance as well as just generally allowing the programmer to automate caching and be done with it (no need to add a do_caching_now call in every new image add method or theme change). I read an old blog post about zenphoto saying that it once was possible to link to arbitrarily sized images to blogs. Ok so the ABILITY to make ''on the fly'' images has a user (blogger) benefit but failing to pre-cache still does not help anyone. It's easy for programmers to confuse coding brilliance with user "features".
I just setup a gallery in gallery3. It went smoothly enough but development direction and support seems fragmented/vague and usability features are maybe a little lacking, at least some including easy image upload features. So, I headed here. Gallery3 probably can't touch how easy it was to copy my whole gallery album structure into ZenPhoto (but then by the time I pre-cached by hand that advantage disappeared, and maybe G3 server add can handle a hierarchy). However, the one thing I like enough about gallery("pre-caching") to DEFINITELY keep me from using zenphoto for now anyway, is being advertised here as a feature and fixing it is being declared as something that isn't really useful.
So why do I think pre-caching is so useful? Because caching takes a ton of memory and/or disk access[* see note below] and web browsing happens in parallel, even with just one user. If you have 100 users browsing pictures and each image takes 5MB of RAM, ok. Now have 100 hundred users each ask to create image caches. To resize the image the full bitmap is stored in memory, often 20 times bigger than the disk file. So now instead of 500MB of ram, that's 10GB of RAM. Well you say reduce the number of apache instances. Why should I? That number of instances is just fine if I serialize the image resizing on my own terms. The argument that ''on the fly'' somehow helps with large servers to handle incremental image additions seems odd. In the end the work required is what it is and delaying it doesn't help the load on the server in any way. Gallery3 keeps track of which files need updating and only incrementally updates those. No problem.
I'm running a small unprofessional server for fun and with other primary purposes. It has fast disk io and more than sufficient bandwidth for its needs to deliver its users full screen images instantly and full size images painlessly. However it is old and is limited in CPU power and even more in RAM(especially being in a VM) and grinds to a near halt or even crashes(ok I need to discipline oom-killer) if I hit the slideshow button with un-cached images. This doesn't happen with gallery3. Maybe I'm completely misdiagnosing this somehow. I would think resource usage would be important to high-powered hosted setups too.
The Solution:
Ok, so you say the community can support this and a plugin exists. Thanks, but I can use themes in G3 and there "precaching" is supported. To do this right some interface should exist that allows themes to declare their pre-caching needs. That is probably so trivial (just a list of resolutions) that a dev could even modify all well known community themes very quickly rather than break compatibility. This should be extremely simple but probably requires core support unless every theme includes their own pre-cacher. I'm sure G3 must have exactly that since different themes do rebuild thumbnails in g3 too in a uniform way. Of course from a programmer perspective ZenPhoto was "not designed to work that way" translated: was designed to allow the programmer to be lazy(no offense, oversimplification acknowledged too) and not need to do that. I love programming simplicity, but it's NOT a user feature.
[*]As for disk access it seems like ZenPhoto is doing a bunch of disk access for these resizes, writing the large bitmap files to /tmp. I don't know enough about imagemagick. It's not clear to me that writing to disk is necessary or ideal. Disk cache might make it not much of an issue though, at least if not too many resizes are happening at once. I'm not sure. Even so, that probably doubles the RAM usage and might be avoidable. I don't claim to begin to know the whole story here, but G3 does seem to me to do the caching faster with less resources. I might try moving my /tmp to a ramdisk and see what happens.
I'm going to plus one this old post and reopen an old and ongoing flame to add to the vote, add my reasoning,add a little comparison to gallery3, and some observations about caching performance in zenphoto.
First it's clear there is little reasonable argument that making a browser ever (even just for the first user) wait to produce an image that could have been produced before is a user feature. Placing the fancy words ''on the fly'' on it doesn't change that fact.
This is primarily a coding feature, not a user feature, and the nice sounding words apply in that context. I suspect it makes it far easier to allow themes to use any size image and not worry about it, and probably other similar brilliance as well as just generally allowing the programmer to automate caching and be done with it (no need to add a do_caching_now call in every new image add method or theme change). I read an old blog post about zenphoto saying that it once was possible to link to arbitrarily sized images to blogs. Ok so the ABILITY to make ''on the fly'' images has a user (blogger) benefit but failing to pre-cache still does not help anyone. It's easy for programmers to confuse coding brilliance with user "features".
I just setup a gallery in gallery3. It went smoothly enough but development direction and support seems fragmented/vague and usability features are maybe a little lacking, at least some including easy image upload features. So, I headed here. Gallery3 probably can't touch how easy it was to copy my whole gallery album structure into ZenPhoto (but then by the time I pre-cached by hand that advantage disappeared, and maybe G3 server add can handle a hierarchy). However, the one thing I like enough about gallery("pre-caching") to DEFINITELY keep me from using zenphoto for now anyway, is being advertised here as a feature and fixing it is being declared as something that isn't really useful.
So why do I think pre-caching is so useful? Because caching takes a ton of memory and/or disk access[* see note below] and web browsing happens in parallel, even with just one user. If you have 100 users browsing pictures and each image takes 5MB of RAM, ok. Now have 100 hundred users each ask to create image caches. To resize the image the full bitmap is stored in memory, often 20 times bigger than the disk file. So now instead of 500MB of ram, that's 10GB of RAM. Well you say reduce the number of apache instances. Why should I? That number of instances is just fine if I serialize the image resizing on my own terms. The argument that ''on the fly'' somehow helps with large servers to handle incremental image additions seems odd. In the end the work required is what it is and delaying it doesn't help the load on the server in any way. Gallery3 keeps track of which files need updating and only incrementally updates those. No problem.
I'm running a small unprofessional server for fun and with other primary purposes. It has fast disk io and more than sufficient bandwidth for its needs to deliver its users full screen images instantly and full size images painlessly. However it is old and is limited in CPU power and even more in RAM(especially being in a VM) and grinds to a near halt or even crashes(ok I need to discipline oom-killer) if I hit the slideshow button with un-cached images. This doesn't happen with gallery3. Maybe I'm completely misdiagnosing this somehow. I would think resource usage would be important to high-powered hosted setups too.
The Solution:
Ok, so you say the community can support this and a plugin exists. Thanks, but I can use themes in G3 and there "precaching" is supported. To do this right some interface should exist that allows themes to declare their pre-caching needs. That is probably so trivial (just a list of resolutions) that a dev could even modify all well known community themes very quickly rather than break compatibility. This should be extremely simple but probably requires core support unless every theme includes their own pre-cacher. I'm sure G3 must have exactly that since different themes do rebuild thumbnails in g3 too in a uniform way. Of course from a programmer perspective ZenPhoto was "not designed to work that way" translated: was designed to allow the programmer to be lazy(no offense, oversimplification acknowledged too) and not need to do that. I love programming simplicity, but it's NOT a user feature.
[*]As for disk access it seems like ZenPhoto is doing a bunch of disk access for these resizes, writing the large bitmap files to /tmp. I don't know enough about imagemagick. It's not clear to me that writing to disk is necessary or ideal. Disk cache might make it not much of an issue though, at least if not too many resizes are happening at once. I'm not sure. Even so, that probably doubles the RAM usage and might be avoidable. I don't claim to begin to know the whole story here, but G3 does seem to me to do the caching faster with less resources. I might try moving my /tmp to a ramdisk and see what happens.
Member
Member
Member
From the beta release notes:
"cacheManager:The pre-caching of images is back as an official plugin. It has been enhanced with options for the sizing of images to create. ALso included are options for automatically purging the caches if content published state changes. [sbillard]"
ah.. ok but this is still a hack even if an official one. If I read this right, the bit about the option for different sizes, there is still no interface between the theme and the pre-cacher. (also would be best if a theme change caused awareness of the necessity, if any, to run it.) This certainly helps tremendously, and maybe it will get better. I'm glad to see official support anyway. I'm also anxious to see how the performance compares to g3 uploads. Soon, but not right now.
"cacheManager:The pre-caching of images is back as an official plugin. It has been enhanced with options for the sizing of images to create. ALso included are options for automatically purging the caches if content published state changes. [sbillard]"
ah.. ok but this is still a hack even if an official one. If I read this right, the bit about the option for different sizes, there is still no interface between the theme and the pre-cacher. (also would be best if a theme change caused awareness of the necessity, if any, to run it.) This certainly helps tremendously, and maybe it will get better. I'm glad to see official support anyway. I'm also anxious to see how the performance compares to g3 uploads. Soon, but not right now.
Member
Maybe you should actually look at the product before you make judgements.
Of course, the "awareness" has always been there--if a a theme needs an image and it is not already in the cache, it is cached. But maybe you are refering to precognition--we have not got that yet.
I cannot comment on the performance. We use the graphics libraries, and they do what they do. Certainly Imagick is proported to be much better than the GD library. The GD library actually is pretty lame in general.
Of course, the "awareness" has always been there--if a a theme needs an image and it is not already in the cache, it is cached. But maybe you are refering to precognition--we have not got that yet.
I cannot comment on the performance. We use the graphics libraries, and they do what they do. Certainly Imagick is proported to be much better than the GD library. The GD library actually is pretty lame in general.
Member
yes "pre" cognition sounds precisely like what I was referring to. Ideally the theme registers its needs when it is activated or options change. Anyway, I'm glad to hear these ideas are not dead. I'll certainly try the beta, but features that aren't there aren't there. I'm on 1.4.2.4 so I'll upgrade.
Administrator
You first need to understand how Zenphoto works. You can either use the default image sizes as set on the general options or the theme can require special sizes (you can have a lot of them depending on the theme complexity).
With 1.4.3 themes can actually register image sizes with the cacheMAnager. Take a look at the official themes like the Zenpage theme for example in the beta about that.
With 1.4.3 themes can actually register image sizes with the cacheMAnager. Take a look at the official themes like the Zenpage theme for example in the beta about that.
Member
Of coure, registering sizes does not cause caching to occur. Since Zenphoto knows nothing of an image until it discovers it, it is pretty much impossible for it to cache the image earlier than this. (Unless of course you activate the `precognition option` above.) Zenphoto does cache the image at its earliest convenience.
Unless you are uploading a considerable number of images or your images are really exessive in size for WEB use the on-the-fly caching works just fine. If you fall into the above category, use the pre-cache feature.
Unless you are uploading a considerable number of images or your images are really exessive in size for WEB use the on-the-fly caching works just fine. If you fall into the above category, use the pre-cache feature.
Member
So I have tried this now...
acrylian:
"You first need to understand how Zenphoto works. "
Well yes and no. First, I think I get the idea of what on the fly means. If a display of an image of a certain size is requested and it doesn't exist it's produced then and there, no? As for *needing* to understand how it works, maybe for making comments about how to solve the problems, but not for requesting how the user interface should act. That's a (volunteer) programmer's problem
. The user's job is to request the behavior The behavior isn't impossible since g3 does it all quite well and the fact that on the fly methods is "how it works" doesn't get in the way of also adding tracking and communication needed to precache. Maybe then it just needs to work other ways too.
"With 1.4.3 themes can actually register image sizes with the cacheMAnager. "
Nice. It looked more like a hand entered table of values, but that's great.
sbillard:
"Of coure, registering sizes does not cause caching to occur. Since Zenphoto knows nothing of an image until it discovers it."
Well, then zenphoto is maybe lacking that functionality. The images are in the albums directory to be discovered. I can see that maybe you want the flexibility for some theme to have a different size for every image (that would be a fun theme for the user to configure). In that case it's hard, and maybe for that case you give up and only let every theme ask for a few sizes that most images use and on the fly the rest. You don't need to give up though. You certainly could have mechanisms for the theme to inform a precacher about what images need what. The only exception I see if the theme actually allows a web page users to demand arbitrary sizes. Then on the fly is on the fly.
So for example in G3 as soon as you change themes a message shows up saying you have X images that need updating. It doesn't just do it though, you still need to go run an admin tool when it says that. When you upload new images, I'm pretty sure then it's automatic though.
Anyway, I tried the beta. The pre-selections are nice and I don't even care if they theme is providing them or a developer is just keeping them updated. What difference does it make to me? Anyway, sadly, I still crashed, after slowly working through a few images! I did look at the code and image magic docs, and yes I see there aren't many choices in how to use imagick. It just does its thing as you say. I do notice that g3 uses the command line "convert" while ZenPhoto uses the php interface. I have no idea why g3 uses convert and wouldn't imagine that there would be any difference since it seems to have all the same options and surely uses the same binaries in the end. So I don't know why G3 works fine and not that slow even on my system and ZenPhoto crashes it. It appears to be related still to parallelism. I see mutliple httpd processes using large amounts of memory and disk I/O. In the end I probably just have to say ZenPhoto looks nice, but it seems my marginal resources just can't quite eek out a win on it and I should be glad that g3 manages to work somehow. I don't count that against ZenPhoto necessarily, just against it in my case. There's a chance I could restrict the number of servers while I precache and then open it up to come back online. I might play with it if I get time.
Thanks for the helpful replies.
acrylian:
"You first need to understand how Zenphoto works. "
Well yes and no. First, I think I get the idea of what on the fly means. If a display of an image of a certain size is requested and it doesn't exist it's produced then and there, no? As for *needing* to understand how it works, maybe for making comments about how to solve the problems, but not for requesting how the user interface should act. That's a (volunteer) programmer's problem
. The user's job is to request the behavior The behavior isn't impossible since g3 does it all quite well and the fact that on the fly methods is "how it works" doesn't get in the way of also adding tracking and communication needed to precache. Maybe then it just needs to work other ways too."With 1.4.3 themes can actually register image sizes with the cacheMAnager. "
Nice. It looked more like a hand entered table of values, but that's great.
sbillard:
"Of coure, registering sizes does not cause caching to occur. Since Zenphoto knows nothing of an image until it discovers it."
Well, then zenphoto is maybe lacking that functionality. The images are in the albums directory to be discovered. I can see that maybe you want the flexibility for some theme to have a different size for every image (that would be a fun theme for the user to configure). In that case it's hard, and maybe for that case you give up and only let every theme ask for a few sizes that most images use and on the fly the rest. You don't need to give up though. You certainly could have mechanisms for the theme to inform a precacher about what images need what. The only exception I see if the theme actually allows a web page users to demand arbitrary sizes. Then on the fly is on the fly.
So for example in G3 as soon as you change themes a message shows up saying you have X images that need updating. It doesn't just do it though, you still need to go run an admin tool when it says that. When you upload new images, I'm pretty sure then it's automatic though.
Anyway, I tried the beta. The pre-selections are nice and I don't even care if they theme is providing them or a developer is just keeping them updated. What difference does it make to me? Anyway, sadly, I still crashed, after slowly working through a few images! I did look at the code and image magic docs, and yes I see there aren't many choices in how to use imagick. It just does its thing as you say. I do notice that g3 uses the command line "convert" while ZenPhoto uses the php interface. I have no idea why g3 uses convert and wouldn't imagine that there would be any difference since it seems to have all the same options and surely uses the same binaries in the end. So I don't know why G3 works fine and not that slow even on my system and ZenPhoto crashes it. It appears to be related still to parallelism. I see mutliple httpd processes using large amounts of memory and disk I/O. In the end I probably just have to say ZenPhoto looks nice, but it seems my marginal resources just can't quite eek out a win on it and I should be glad that g3 manages to work somehow. I don't count that against ZenPhoto necessarily, just against it in my case. There's a chance I could restrict the number of servers while I precache and then open it up to come back online. I might play with it if I get time.
Thanks for the helpful replies.
Member
Oh as for the large size images. I don't think it's fair to say they are inappropriate for a web gallery. I've seen where devs here said that before and always found that odd. My server can easily handle delivering them and delivering them quickly and I want users to be able to just click from the album to get them. How can one say that's not appropriate if 1) it's easily possible and 2) it's what I and the users want?
So the only problem is image resizing. I just tried to convert an image by hand with convert. I get one process using a typically large amount of memory, but only one. And it finishes in a second or two without destroying my system at all. So this should be possible, as I already knew.
So the only problem is image resizing. I just tried to convert an image by hand with convert. I get one process using a typically large amount of memory, but only one. And it finishes in a second or two without destroying my system at all. So this should be possible, as I already knew.