Member
Hi!
I know there already are some discussions (like [this](https://forum.zenphoto.org/discussion/1409095/) or [this](https://forum.zenphoto.org/discussion/1397124/)) concerning change dates of images, but I haven't found a good answer to this topic yet.
My goal is to get a list of all images changed since a certain timestamp to sync these with a remote CMS. Those changes should include editing metadata, refreshing metadata from file (EXIF, IPTC, …) and filesystem operations like moving images to another album.
Looking at the database table `images` I noticed the fields `date` and `mtime`, but these seem to correspond to the date/time an image was taken and the modification timestamp of the file respectively. Obviously metadata fields like `EXIFDateTime` aren't useable for this purpose either. The field `publishdate` sounded most promising, but is only used to schedule publishing an image in the future.
I also thought about using `custom_data`, but then it couldn't be used for anything else and couldn't even store serialized data because that would prohibit sorting.
Given that this seems to be a common question and that the topic comes up every now and then: Is there a best practice approach you guys would recommend? Is there anything planned for a future release?
Tobias
I know there already are some discussions (like [this](https://forum.zenphoto.org/discussion/1409095/) or [this](https://forum.zenphoto.org/discussion/1397124/)) concerning change dates of images, but I haven't found a good answer to this topic yet.
My goal is to get a list of all images changed since a certain timestamp to sync these with a remote CMS. Those changes should include editing metadata, refreshing metadata from file (EXIF, IPTC, …) and filesystem operations like moving images to another album.
Looking at the database table `images` I noticed the fields `date` and `mtime`, but these seem to correspond to the date/time an image was taken and the modification timestamp of the file respectively. Obviously metadata fields like `EXIFDateTime` aren't useable for this purpose either. The field `publishdate` sounded most promising, but is only used to schedule publishing an image in the future.
I also thought about using `custom_data`, but then it couldn't be used for anything else and couldn't even store serialized data because that would prohibit sorting.
Given that this seems to be a common question and that the topic comes up every now and then: Is there a best practice approach you guys would recommend? Is there anything planned for a future release?
Tobias
Administrator
mtime should actually be the image file date if was uploaded. Not sure right now what happens if you upload an image with the exact same name to replace it (via ftp for example). Then Zenphoto would not treat it as new and might not change the mtime value.
The question is also what is the change date here. Change to the image file or changes to the titlte, description on the Zenphoot backend. For the latter there are plans to add a last changedate as Zenpage items have already.
You can very well use the `custom data` field as it is an all purpose field. You can also use one of the other standard fields if you know you don't use them otherwise.
There is also a very inconvenient way to add further extra custom fields using the the fieldextender plugin. It requires some coding, instructioms are in `zp-extensions/common/fieldextender`. Handling of such extra custom fields will be largely reworked in the future (and may not be directly compatible - remember our talk about moving image meta data to a separate table).
The question is also what is the change date here. Change to the image file or changes to the titlte, description on the Zenphoot backend. For the latter there are plans to add a last changedate as Zenpage items have already.
You can very well use the `custom data` field as it is an all purpose field. You can also use one of the other standard fields if you know you don't use them otherwise.
There is also a very inconvenient way to add further extra custom fields using the the fieldextender plugin. It requires some coding, instructioms are in `zp-extensions/common/fieldextender`. Handling of such extra custom fields will be largely reworked in the future (and may not be directly compatible - remember our talk about moving image meta data to a separate table).
Member
I have to revisit this topic as I have finally found some time to move forward with the sync plugin I mentioned.
I decided to use the field `publishdate` and update it to a current timestamp whenever an image is changed (metadata & filesystem operations). I might change that to `changedate` once that field is available, but for now `publishdate` seems to be the best approach for me.
I'm going to use the [Root_class object filter](https://www.zenphoto.org/news/zenphoto-plugin-architecture/#rootclass) `save_object` for this purpose, but the user guide does not explain the first (boolean) parameter of this filter. When invoked by `PersistentObject->save()` the parameter is always supplied with `true`, but I'd like to understand it's meaning in case it will be used more "dynamically" in the future.
Another thing I noticed is that this filter is only invoked right at the end of `PersistentObject->save()` so in order to change the data written to the database I have to update it and call `save()` again. For this to work I need to un-register my filter plugin first of course to prevent an endless loop. After saving I have to re-register it again. Any idea on how to improve that?
I decided to use the field `publishdate` and update it to a current timestamp whenever an image is changed (metadata & filesystem operations). I might change that to `changedate` once that field is available, but for now `publishdate` seems to be the best approach for me.
I'm going to use the [Root_class object filter](https://www.zenphoto.org/news/zenphoto-plugin-architecture/#rootclass) `save_object` for this purpose, but the user guide does not explain the first (boolean) parameter of this filter. When invoked by `PersistentObject->save()` the parameter is always supplied with `true`, but I'd like to understand it's meaning in case it will be used more "dynamically" in the future.
Another thing I noticed is that this filter is only invoked right at the end of `PersistentObject->save()` so in order to change the data written to the database I have to update it and call `save()` again. For this to work I need to un-register my filter plugin first of course to prevent an endless loop. After saving I have to re-register it again. Any idea on how to improve that?
Administrator
Can't answer about `save_object` filter parameters as I don't know offhand.
You can remove a filter using `zp_remove_filter()`. It will be temprary since on page reload the normal process will re-register. Not sure I understand the loop issue but I am not into your problem. W
Just to note `publishdate` is a standard colum of the images table and is setup for `datetime` (`YYYY-MM-DD HH:MM:SS`) actually and not (UNIX) time stamps. Just to avoid confusion.
If you are really into have a changedate you could also use the general purpose `plugin_storage` table without needing to re-use "publishdate" for something else or even create a new column. (Not sure why I forgot to mention that apparently).
You can remove a filter using `zp_remove_filter()`. It will be temprary since on page reload the normal process will re-register. Not sure I understand the loop issue but I am not into your problem. W
Just to note `publishdate` is a standard colum of the images table and is setup for `datetime` (`YYYY-MM-DD HH:MM:SS`) actually and not (UNIX) time stamps. Just to avoid confusion.
If you are really into have a changedate you could also use the general purpose `plugin_storage` table without needing to re-use "publishdate" for something else or even create a new column. (Not sure why I forgot to mention that apparently).
Member
The loop issues will come up if you try something like this:
function myFilter($bool, $obj) { $obj->set(...); $obj->save(); }
zp_register_filter('save_object', 'myFilter');
$mediaObject->save();
This will cause `save()` to call `myFilter()` which will call `save()` which will call `myFilter()` which will call `save()` and so on. So you have to un-register the filter inside `myFilter()` before calling `save()` in order to prevent the callback. Not the prettiest of solutions, but it should work.
Thanks for mentioning `plugin_storage`. I will have a look at it, but it might not deliver best performance if I add an entry for every image and filter over `data` (text).
(Is there an index for `plugin_storage.data` or `images.publishdate`?)
function myFilter($bool, $obj) { $obj->set(...); $obj->save(); }
zp_register_filter('save_object', 'myFilter');
$mediaObject->save();
This will cause `save()` to call `myFilter()` which will call `save()` which will call `myFilter()` which will call `save()` and so on. So you have to un-register the filter inside `myFilter()` before calling `save()` in order to prevent the callback. Not the prettiest of solutions, but it should work.
Thanks for mentioning `plugin_storage`. I will have a look at it, but it might not deliver best performance if I add an entry for every image and filter over `data` (text).
(Is there an index for `plugin_storage.data` or `images.publishdate`?)
Administrator
Hm, I haven't looked at the code yet. But you should not need to call `$obj->save()` within your filter call. It should only modify the object before saving it by the normal process.
If it does not do that the filter might be implemented wrong for ages and no one noticed…
> (Is there an index for plugin_storage.data or images.publishdate?)
No,actually no primariy columns. `plugin_storage` has indices on `type` and `aux`.
And the images table only on id, albumid and filename as those are the primary fields.
If it does not do that the filter might be implemented wrong for ages and no one noticed…
> (Is there an index for plugin_storage.data or images.publishdate?)
No,actually no primariy columns. `plugin_storage` has indices on `type` and `aux`.
And the images table only on id, albumid and filename as those are the primary fields.
Administrator
Just have take a quick look as the `save()` method. It seems the filter is called after any db action is performed. So the data modified indeed will never be updated without $obj->save(). It's not even added to the cache as the filter does not return anything. At first glance it look like this is somehow not right at all. Have to search if this filter is actually used by anything in the core and if how…
Member
Administrator
I also have quickly looked at the new/update_* filters for Zenpage items and to `new_image/album`. These all do pass the object through so functions hooked can modify it but all are called after it is already saved. So also need to do an additional save if needed.
Therefore thinking of introducing additional filter hooks before saving objects and also to not break any existing usages.
> but the user guide does not explain the first (boolean) parameter of this filter.
It is just useless. Probably meant as a check to be false on some occasion. Generally zp_apply_filter returns the first parameter value (unless the function hook does different things).
Btw, did you try to use the `new_image` or `Image_instaniate` filter instead of the `save_object` one in the root class? That should not create a loop.
Therefore thinking of introducing additional filter hooks before saving objects and also to not break any existing usages.
> but the user guide does not explain the first (boolean) parameter of this filter.
It is just useless. Probably meant as a check to be false on some occasion. Generally zp_apply_filter returns the first parameter value (unless the function hook does different things).
Btw, did you try to use the `new_image` or `Image_instaniate` filter instead of the `save_object` one in the root class? That should not create a loop.
Member
So `zp_apply_filter()` cannot even pass the object to a filter function because only the first parameter is passed, correct?
zp_apply_filter('save_object', true, $this);
function zp_apply_filter($hook, $value = '') {
[…]
$args[1] = $value;
$new_value = call_user_func_array($the_['function'], array_slice($args, 1));
[…]
}
Introducing additional filter hooks sounds like solid idea. Existing ones can still be useful to notify plugins of events like "object (successfully) saved".
I cannot use the filter hooks `new_image` and `image_instantiate` for my plugin because it needs to know whenever an image object is changed. At a glance it looks like `new_image` is only called for new discovered images. So all changes that are made later on like editing the title will not invoke my filter to update the change date. `image_instantiate` on the other hand seems to be invoked whenever a runtime [image] object is created. So this will be called way too often and also although no change was made to the image.
zp_apply_filter('save_object', true, $this);
function zp_apply_filter($hook, $value = '') {
[…]
$args[1] = $value;
$new_value = call_user_func_array($the_['function'], array_slice($args, 1));
[…]
}
Introducing additional filter hooks sounds like solid idea. Existing ones can still be useful to notify plugins of events like "object (successfully) saved".
I cannot use the filter hooks `new_image` and `image_instantiate` for my plugin because it needs to know whenever an image object is changed. At a glance it looks like `new_image` is only called for new discovered images. So all changes that are made later on like editing the title will not invoke my filter to update the change date. `image_instantiate` on the other hand seems to be invoked whenever a runtime [image] object is created. So this will be called way too often and also although no change was made to the image.
Administrator
> So zp_apply_filter() cannot even pass the object to a filter function because only the first parameter is passed, correct?
No, the first is actually just default and passed through if nothing is hooked. Actually it returns what ever the attached function/method returns if it returns anything. So it could return the modified object even as the 2nd parameter if it was returned by doing `$this = zp_apply_filter('save_object', true, $this);` for example.
Thanks for the explaination why you have to use that filter. Makes absolutely sense. Since the root `save()` method does differ between "new" and "update" two new hooks could be added. Right before the foreach loops for preparing the db data occurs would probably be the best place. What do you think? Then I would the next days add them to to the 1.5.2a and you could test them.
(Sidenote: just tried the Zenpage ones and although they don't return the object either, they can be used to modify).
No, the first is actually just default and passed through if nothing is hooked. Actually it returns what ever the attached function/method returns if it returns anything. So it could return the modified object even as the 2nd parameter if it was returned by doing `$this = zp_apply_filter('save_object', true, $this);` for example.
Thanks for the explaination why you have to use that filter. Makes absolutely sense. Since the root `save()` method does differ between "new" and "update" two new hooks could be added. Right before the foreach loops for preparing the db data occurs would probably be the best place. What do you think? Then I would the next days add them to to the 1.5.2a and you could test them.
(Sidenote: just tried the Zenpage ones and although they don't return the object either, they can be used to modify).
Administrator
Just took another look. Zenpage items have said "new/update" filter if articles, pages or categories are saved on the backend.
Images and albums don't have such named filters but actually the two existing `save_image_utilities_data` and `'save_album_utilities_data` despite their name can do the same. They don't return the object but they are called right before calling `save()` on the objects.
Unless you really need to do this on class level those probably could help you, too? Or did you try those already?
Also would a new column `lastchange` (and matching `lastchangeowner`) help as Zenpage articles and pages have it? We thought of adding those for images and albums anyway to match the other items.
Images and albums don't have such named filters but actually the two existing `save_image_utilities_data` and `'save_album_utilities_data` despite their name can do the same. They don't return the object but they are called right before calling `save()` on the objects.
Unless you really need to do this on class level those probably could help you, too? Or did you try those already?
Also would a new column `lastchange` (and matching `lastchangeowner`) help as Zenpage articles and pages have it? We thought of adding those for images and albums anyway to match the other items.
Administrator
I was wrong, seems a filter call like `zp_apply_filter('somefilter', $value, $obj);` would indeed update `$obj` if the function attached modifies it. No need to return or `save` here. Somehow this detail escaped me right now although lots of filters are setup… You could try that with the filters I mentioned above. The `save_object` filter still is not placed correctly for the actual saving IMHO.
Member
I'm sorry for the late reply! Had a busy week at work.
I had to debug `zp_apply_filter()` to get an idea of how exactly it is doing what it's doing.
Setting `$value` as `$args[1]` kinda threw me off as well as `array_slice($args, 1)`.
(The first one is only required if `$value` was not supplied to `zp_apply_filter()`. The second one is used to truncate `$hook` from the list of arguments passed to the filter function.)
> Since the root `save()` method does differ between "new" and "update" two new hooks could be added. Right before the foreach loops for preparing the db data occurs would probably be the best place. What do you think?
That's exactly what I thought.
> Then I would the next days add them to to the 1.5.2a and you could test them.
That would be awesome!
miley:
> Images and albums don't have such named filters but actually the two existing `save_image_utilities_data` and `save_album_utilities_data` despite their name can do the same. […]
> Unless you really need to do this on class level those probably could help you, too? Or did you try those already?
I haven't tried already, but I'm not sure using them would be good idea. The description in the user guide sounds like they are intended for completely different tasks. So although they might work now, I think it would be better to use a hook that is intended to modify an object right before saving.
> Also would a new column `lastchange` (and matching `lastchangeowner`) help as Zenpage articles and pages have it? We thought of adding those for images and albums anyway to match the other items.
Yes, that would be exactly the thing I was looking for. I can live with `publishdate` for the time being and migrate my data once those columns will be added (if they are to be).
> I was wrong, seems a filter call like `zp_apply_filter('somefilter', $value, $obj);` would indeed update `$obj` if the function attached modifies it.
Are you sure this isn't depending on whether `$obj` is an Object or a variable? Maybe that would influence if it is passed as call-by-reference or call-by-value and hence whether you are modifing the original section of memory or just a copy of that.
I had to debug `zp_apply_filter()` to get an idea of how exactly it is doing what it's doing.
Setting `$value` as `$args[1]` kinda threw me off as well as `array_slice($args, 1)`.
(The first one is only required if `$value` was not supplied to `zp_apply_filter()`. The second one is used to truncate `$hook` from the list of arguments passed to the filter function.)
> Since the root `save()` method does differ between "new" and "update" two new hooks could be added. Right before the foreach loops for preparing the db data occurs would probably be the best place. What do you think?
That's exactly what I thought.
> Then I would the next days add them to to the 1.5.2a and you could test them.
That would be awesome!
miley:> Images and albums don't have such named filters but actually the two existing `save_image_utilities_data` and `save_album_utilities_data` despite their name can do the same. […]
> Unless you really need to do this on class level those probably could help you, too? Or did you try those already?
I haven't tried already, but I'm not sure using them would be good idea. The description in the user guide sounds like they are intended for completely different tasks. So although they might work now, I think it would be better to use a hook that is intended to modify an object right before saving.
> Also would a new column `lastchange` (and matching `lastchangeowner`) help as Zenpage articles and pages have it? We thought of adding those for images and albums anyway to match the other items.
Yes, that would be exactly the thing I was looking for. I can live with `publishdate` for the time being and migrate my data once those columns will be added (if they are to be).
> I was wrong, seems a filter call like `zp_apply_filter('somefilter', $value, $obj);` would indeed update `$obj` if the function attached modifies it.
Are you sure this isn't depending on whether `$obj` is an Object or a variable? Maybe that would influence if it is passed as call-by-reference or call-by-value and hence whether you are modifing the original section of memory or just a copy of that.
Administrator
> … but actually the two existing save_image_utilities_data and save_album_utilities_data
Well, they are in the same place as individual "save" ones would be like:
` zp_apply_filter('save_image_utilities_data', $image, $index);
$image->save();`
Despite their name they are not limited to "utilities data" (which refers to plugins adding extras to the backend right sidebar). Please try them nevertheless. No need to add things if these work for your purpose
> Are you sure this isn't depending on whether $obj is an Object or a variable? Maybe that would influence if it is passed as call-by-reference or call-by-value and hence whether you are modifing the original section of memory or just a copy of that.
I thought too but I didn't find any passing by reference directly yet. At least with the ZEnpage filter it works as I see saved changes without doing an extra save within the filter hook.
> I can live with publishdate for the time being
Okay, I will consider to add these in 1.5.2a soon. Publishdate is really meant for scheduled publishing.
You see ZP is a complex tool that even we as devs don't always remember everything ;-)
Well, they are in the same place as individual "save" ones would be like:
` zp_apply_filter('save_image_utilities_data', $image, $index);
$image->save();`
Despite their name they are not limited to "utilities data" (which refers to plugins adding extras to the backend right sidebar). Please try them nevertheless. No need to add things if these work for your purpose
> Are you sure this isn't depending on whether $obj is an Object or a variable? Maybe that would influence if it is passed as call-by-reference or call-by-value and hence whether you are modifing the original section of memory or just a copy of that.
I thought too but I didn't find any passing by reference directly yet. At least with the ZEnpage filter it works as I see saved changes without doing an extra save within the filter hook.
> I can live with publishdate for the time being
Okay, I will consider to add these in 1.5.2a soon. Publishdate is really meant for scheduled publishing.
You see ZP is a complex tool that even we as devs don't always remember everything ;-)
Member
I tried all three hooks and while they work as expected when editing the image data (like title, description etc.), they don't seem to be invoked when refreshing metadata or moving an image to a different folder.
I was expecting `PersistentObject->save()` to be called on these occasions too because both operations will result in changes on the `images` database table.
Any idea why it doesn't work or what I could try instead?
I was expecting `PersistentObject->save()` to be called on these occasions too because both operations will result in changes on the `images` database table.
Any idea why it doesn't work or what I could try instead?
Administrator
Yes, `save_image_utilities_data` is a backend only so onl applies when saving changes there.
The `updateMetaData()` method of the image class does have its own filter `image_metadata`. The meta refresh is actually handled by the gallery class method `garbageCollect()` which matches the database with the file system and it does call that method and does call `save()` on the image object actually. So the `save_object` filter should actually be used.
The `updateMetaData()` method of the image class does have its own filter `image_metadata`. The meta refresh is actually handled by the gallery class method `garbageCollect()` which matches the database with the file system and it does call that method and does call `save()` on the image object actually. So the `save_object` filter should actually be used.
Member
I created `Image->save()` and added a `debugLog()` command to it. Then I activated `DEBUG_FILTERS` in `global-definitions.php`. This is what happens with my filter registered to hook `save_object` and move an image from one folder to another:
[start of request]
{12839:Sun, 17 Feb 2019 16:04:26 GMT}
[Zenphoto starting to register all the filters]
[…]
{12839:Sun, 17 Feb 2019 16:04:26 GMT}
media_change_date.php=>media_change_date::updateChangeDate registered to save_object at priority 5
[…]
{12839:Sun, 17 Feb 2019 16:04:26 GMT}
Image:ave() was called from:
#0 zp-core/admin-functions.php(2557): Image->save()
#1 zp-core/admin-edit.php(278): processImageEdit(Object(Image), 0)
#2 {main}
{12839:Sun, 17 Feb 2019 16:04:26 GMT}
Apply filters for save_image_utilities_data
exiftool::putEXIF
[This another plugin of mine. Proofs filters get applied.]
{12839:Sun, 17 Feb 2019 16:04:26 GMT}
Image:ave() was called from:
#0 zp-core/admin-functions.php(2557): Image->save()
#1 zp-core/admin-edit.php(278): processImageEdit(Object(Image), 1)
#2 {main}
{12839:Sun, 17 Feb 2019 16:04:26 GMT}
Image:ave() was called from:
#0 zp-core/class-image.php(857): Image->save()
#1 zp-core/admin-functions.php(2564): Image->move(Object(Album))
#2 zp-core/admin-edit.php(278): processImageEdit(Object(Image), 1)
#3 {main}
{31647:Sun, 17 Feb 2019 16:04:26 GMT}
[Zenphoto starting to register all the filters again after roundtrip]
[…]
So although the filter is registered and `Image->save()` gets called multiple times the filter is not applied.
[start of request]
{12839:Sun, 17 Feb 2019 16:04:26 GMT}
[Zenphoto starting to register all the filters]
[…]
{12839:Sun, 17 Feb 2019 16:04:26 GMT}
media_change_date.php=>media_change_date::updateChangeDate registered to save_object at priority 5
[…]
{12839:Sun, 17 Feb 2019 16:04:26 GMT}
Image:
ave() was called from:#0 zp-core/admin-functions.php(2557): Image->save()
#1 zp-core/admin-edit.php(278): processImageEdit(Object(Image), 0)
#2 {main}
{12839:Sun, 17 Feb 2019 16:04:26 GMT}
Apply filters for save_image_utilities_data
exiftool::putEXIF
[This another plugin of mine. Proofs filters get applied.]
{12839:Sun, 17 Feb 2019 16:04:26 GMT}
Image:
ave() was called from:#0 zp-core/admin-functions.php(2557): Image->save()
#1 zp-core/admin-edit.php(278): processImageEdit(Object(Image), 1)
#2 {main}
{12839:Sun, 17 Feb 2019 16:04:26 GMT}
Image:
ave() was called from:#0 zp-core/class-image.php(857): Image->save()
#1 zp-core/admin-functions.php(2564): Image->move(Object(Album))
#2 zp-core/admin-edit.php(278): processImageEdit(Object(Image), 1)
#3 {main}
{31647:Sun, 17 Feb 2019 16:04:26 GMT}
[Zenphoto starting to register all the filters again after roundtrip]
[…]
So although the filter is registered and `Image->save()` gets called multiple times the filter is not applied.
Administrator
Thanks for the tests. The image class `move()` method actually does call `save()` if there was something to move actually. So this will need some more investigation I fear.
If we add a new column for last change I would implement that they are automatically set with the change date automatically. But nevertheless the filter probably should work as at least we expect it to do.
If we add a new column for last change I would implement that they are automatically set with the change date automatically. But nevertheless the filter probably should work as at least we expect it to do.
Administrator
`lastchange` and `lastchangeuser` are now implemented in the support build. `lastchange` is set to the current whenever an object is saved automatically. Regardless if by code or if it is from the backend pages via direct admin request. In that case the `lastchangeuser` is set additionally, otherwise it is empty (backend reads "Code request" for now).