ZenphotoCMS Forum
Unicode characters in description - Printable Version

+- ZenphotoCMS Forum (https://forum.zenphoto.org)
+-- Forum: Support (https://forum.zenphoto.org/forum-1.html)
+--- Forum: General support (https://forum.zenphoto.org/forum-4.html)
+--- Thread: Unicode characters in description (/thread-12249.html)



Unicode characters in description - Aasemoon - 03-05-2018

Hi there,

First pardon me if the solution to this has already been mentioned somewhere, I was unable to find it.

I've noticed that I can't have unicode characters in the description of my images or albums, if I do then the description disappears. This is while I can use unicode characters in other places, such as tags for instance. Is there a way to repair this?

Cheers!




Unicode characters in description - acrylian - 03-05-2018

What unicode characters are you referring to? Umlauts like öäü and other special chars should work as Zenphoto uses UTF8 for database collation (check if you do use this in case you have an older install or ugraded an older). Multibyte characters like emojis indeed don't, this require some database to support UTF8_mb4 work we haven't done yet. You can change that via phpmyadmin if you need but it may have side effects with existing content because of the column sizes changes. you will need at least MySQL 5.5.3+ to do so, too.

There is also a ticket for this already. We might not get this into 1.x anymore.




Unicode characters in description - sbillard - 03-05-2018

to save you the search, this is the ticket: https://github.com/zenphoto/zenphoto/issues/1094




Unicode characters in description - acrylian - 03-05-2018

And to save from some confusion, that "Zenphoto" referenced on that ticket is a completely separate project.




Unicode characters in description - Aasemoon - 06-05-2018

Hi there, thanks for the response. [And sorry for my late one, never received a notification...]
What I noticed always results in my descriptions disappearing, is the "α" character. I'm sure I had others in the past but I'm not sure what they were. Would this require the "UTF8_mb4" as well?




Unicode characters in description - acrylian - 06-05-2018

Standard UTF8 (utf8_unicode_ci) should do for this. I at least cannot not reproduce any descriptions disappearing wiht that encoding and that alpha special character.

Current Zenphoto versions should actually set up UTF8 but if your install exists for some years already it might still use some other encoding. Also check if your server itself usese utf8 for transferring data (POST/GET) as it is not only the MySQL storage itself.




Unicode characters in description - MarkRH - 09-05-2018

Hmm.. both my blog and gallery databases are set to utf8_unicode_ci and emojis like the alien and pizza work in my blog but here it makes the description go poof.

Well, I guess my blog saves the emojis in the database like "Pizza: ? " If I copy this directly into the Zenphoto database field, it blanks it on the site.




Unicode characters in description - sbillard - 10-05-2018

Some emojis are simple UTF-8 but others, particularly more recently introduced ones, require UTF-8 multi-byte 4. The Pizza emoji is "\xf0\x9f\x8d\x95" which requires the 4 byte representation.

So, bottom line, if a character is causing the database string to be truncated, it is probably an MB4 character.




Unicode characters in description - MarkRH - 11-05-2018

Ok, this is how my blog stores the pizza emoji: ? so it's storing it in the database as an HTML entity, not as a single character perse. That's why it still works.




Unicode characters in description - sbillard - 15-05-2018

If you are using tinyMCE you can change the configuration to have it do this conversion as well: https://www.tinymce.com/docs-3x/reference/configuration/Configuration3x@entity_encoding/




Unicode characters in description - acrylian - 16-05-2018

This is the correct link for the current TinyMCE we use:
https://www.tinymce.com/docs/configure/content-filtering/#encoding

Our current included config files use entity_encoding: 'raw', by default.




Unicode characters in description - MarkRH - 23-05-2018

Yep, changing them all to entity_encoding: 'numeric', seems to make it work.




Unicode characters in description - acrylian - 24-05-2018

Okay, we will add options to the tinymce plugin for this.