Unicode characters in description

Hi there,

First pardon me if the solution to this has already been mentioned somewhere, I was unable to find it.

I've noticed that I can't have unicode characters in the description of my images or albums, if I do then the description disappears. This is while I can use unicode characters in other places, such as tags for instance. Is there a way to repair this?

Cheers!

Tags:

Comments

  • acrylian Administrator, Developer
    edited May 2018

    What unicode characters are you referring to? Umlauts like öäü and other special chars should work as Zenphoto uses UTF8 for database collation (check if you do use this in case you have an older install or ugraded an older). Multibyte characters like emojis indeed don't, this require some database to support UTF8_mb4 work we haven't done yet. You can change that via phpmyadmin if you need but it may have side effects with existing content because of the column sizes changes. you will need at least MySQL 5.5.3+ to do so, too.

    There is also a ticket for this already. We might not get this into 1.x anymore.

  • to save you the search, this is the ticket: https://github.com/zenphoto/zenphoto/issues/1094

  • acrylian Administrator, Developer

    And to save from some confusion, that "Zenphoto" referenced on that ticket is a completely separate project.

  • Hi there, thanks for the response. [And sorry for my late one, never received a notification...]
    What I noticed always results in my descriptions disappearing, is the "α" character. I'm sure I had others in the past but I'm not sure what they were. Would this require the "UTF8_mb4" as well?

  • acrylian Administrator, Developer
    edited May 2018

    Standard UTF8 (utf8_unicode_ci) should do for this. I at least cannot not reproduce any descriptions disappearing wiht that encoding and that alpha special character.

    Current Zenphoto versions should actually set up UTF8 but if your install exists for some years already it might still use some other encoding. Also check if your server itself usese utf8 for transferring data (POST/GET) as it is not only the MySQL storage itself.

  • MarkRH Member
    edited May 2018

    Hmm.. both my blog and gallery databases are set to utf8_unicode_ci and emojis like the alien and pizza work in my blog but here it makes the description go poof.

    Well, I guess my blog saves the emojis in the database like "Pizza: 🍕 " If I copy this directly into the Zenphoto database field, it blanks it on the site.

  • Some emojis are simple UTF-8 but others, particularly more recently introduced ones, require UTF-8 multi-byte 4. The Pizza emoji is "\xf0\x9f\x8d\x95" which requires the 4 byte representation.

    So, bottom line, if a character is causing the database string to be truncated, it is probably an MB4 character.

  • MarkRH Member

    Ok, this is how my blog stores the pizza emoji: 🍕 so it's storing it in the database as an HTML entity, not as a single character perse. That's why it still works.

  • If you are using tinyMCE you can change the configuration to have it do this conversion as well: https://www.tinymce.com/docs-3x/reference/configuration/Configuration3x@entity_encoding/

  • acrylian Administrator, Developer

    This is the correct link for the current TinyMCE we use:
    https://www.tinymce.com/docs/configure/content-filtering/#encoding

    Our current included config files use entity_encoding: 'raw', by default.

  • MarkRH Member

    Yep, changing them all to entity_encoding: 'numeric', seems to make it work.

  • acrylian Administrator, Developer

    Okay, we will add options to the tinymce plugin for this.

Sign In or Register to comment.