set imageData pulled from EXIF - reverse geocoding

tunafish

Ok so I understand that when I upload a new image and it contains GPS data in the EXIF at some point it gets saved in the fields `EXIFGPSLatitude, EXIFGPSLatitudeRef, EXIFGPSLongitude, EXIFGPSLongitudeRef`
I would like to know when and how this happens.

What I would like to achieve (maybe it's even candidate for a feature request?) is that at this point when GPS data is found the `location, city, state, country` fields get automatically set in the database,. multilanguage.

I did some test with Google's reverse geocoding API and with looping over the `generateLanguageList()` function this data can be queried with a Curl.

Here is a function that I can call from image.php

`
function reverseGeoCode() {

$location = getImageData('location');
$city = getImageData('city');
$state = getImageData('state');
$country = getImageData('country');

// are the data fields location, city, state, country empty in the database?
if (empty($location) && empty($city) && empty($state) && empty($country)) {

$lat = getImageData('EXIFGPSLatitude');
$lng = getImageData('EXIFGPSLongitude');

// is GPS data found in the EXIF?
if (!empty($lat) && !empty($lng)) {

(getImageData('EXIFGPSLatitudeRef') == "N") ? $lat_ref = "" : $lat_ref = "-";
(getImageData('EXIFGPSLongitudeRef') == "E") ? $lng_ref = "" : $lng_ref = "-";
$latlng = "$lat_ref$lat,$lng_ref$lng"; // string for Google

// multilanguage arrays for the fields
$arr_location = $arr_city = $arr_state = $arr_country = array();

// query Google Geocoding API to reverse geocode for each language
// http://code.google.com/apis/maps/documentation/geocoding/#ReverseGeocoding
$_languages = generateLanguageList();
foreach ($_languages as $text => $locale) {

$locale_code = substr($locale, 0, 2);
$geocodeURL = "http://maps.googleapis.com/maps/api/geocode/json?latlng=$latlng&language=$locale_code&sensor=false";

// Curl them (not tested on performance and delays yet)
$ch = curl_init($geocodeURL);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$result = curl_exec($ch);
$httpCode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
curl_close($ch);

// continue only if we get the correct response
if ($httpCode == 200) {
$geocode = json_decode($result);
// the data fields we are after
$g_location = $geocode->results[0]->address_components[0]->long_name;
$g_city = $geocode->results[0]->address_components[1]->long_name;
$g_province = $geocode->results[0]->address_components[2]->long_name;
$g_region = $geocode->results[0]->address_components[3]->long_name;
$g_country = $geocode->results[0]->address_components[4]->long_name;

$geo_status = $geocode->status;

// add the response data to the multilanguage arrays
$arr_location[$locale] = $g_location;
$arr_city[$locale] = $g_city;
$arr_state[$locale] = $g_province;
$arr_country[$locale] = $g_country;
}
/*
else {
$geo_status = "HTTP_FAIL_$httpCode";
$geo_str = "Failed: $geo_status";
}*/

}

// a static map image
// echo "";

// serialze the multilanguage arrays
echo "table location : " . serialize($arr_location) . "
";
echo "table city : " . serialize($arr_city) . "
";
echo "table state : " . serialize($arr_state) . "
";
echo "table country : " . serialize($arr_country) . "
";

}
}
}
`
So how and when can I set these fields? Or can I just do them from image.php?
What if I do a "refresh metadata" in the admin? Let's say I'll update the GPS data from an image. How can I update the other fields?
Any more points a need to be aware of? Will Curling like that work?

Cheers

acrylian

My collegue is the meta data expert. But generally the reading and populating of the fields in the db happens on image discovery. It depends of course which metadata fields you enabled on the options.

You could probably do that on the theme side (image.php) but there should be filters for the backend (don't remember exactly right now, best you take a look at the plugin tutorial).

tunafish

OK how to do it is fairly simple, I create an $imageobject and than

`
$imageobject->setLocation(serialize($arr_location));
$imageobject->setCity(serialize($arr_city));
$imageobject->setState(serialize($arr_state));
$imageobject->setCountry(serialize($arr_country));

$imageobject->save();
`
Works fine, the only thing is when and where.
Right now I do it from image.php with `reverseGeoCode()` written in themes/mytheme/functions.php
The advantage is that I don't have to mess around with core files. The only problem is that the image needs one first 'hit' before the data is written.
And than there is the "update metadata" thing, it will not update the fields because of my `if (empty($location) && empty($city) && empty($state) && empty($country))` statement

Any suggestions or thoughts very welcome!

acrylian

Generally the metadata plugin overwrites on update any existing content.

We have filter call `zp_apply_filter('image_instantiate', $image);` within the newImage() function to create an image where $image is the object. You probably can use that.

sbillard

There is actually a document on the filters that you could look at. As well as the xmpMetadata plugin that you could review as an example. `image_metadata` is of course the filter you wish to attach.

tunafish

So I could use this function from a plugin? Cool bananas!
Cheers guys. I am off for a few days but could maybe share such a plugin when I am back (after review of course :-)

sbillard

We will look forward to this plugin.

tunafish

Hi, okay I have a first version, please let me know what you guys think...

Performance wise all seems to work with the cURL's, although a "refresh metadata" on a database with a LOT of images will make a lot of calls to Google. Each image multiplied by the number of active languages. Google lets you do query limit of 2,500 geolocation requests per day: http://code.google.com/apis/maps/documentation/geocoding/#Limits

I also need to test what happens on an installation with the multi language not active btw.

I have not tested it against the xmpMetadata plugin. I think you can set some sort of priority right? Is this necessary or will one just override the fields in the database?

What I would also like to integrate is some sort of lock if the `location`, `city`, `state` & `country` fields are already filled out. For example I reverse_geocode the image(s) and than correct the `location` field with admin-edit.php. The next time I do a "refresh metadata" the fields will be overriden.. I would like some option to lock a particular field from a particular image. I think it's especially useful for the `location` field.
For example, instead of the geocoded street name I could want something like "old town".
Any ideas very welcome.

Anyway, here is the plugin attempt:

`
<?php

/**
* @author Anton Puttemans (tunafish)
* @package plugins
*/

$plugin_is_filter = 9|CLASS_PLUGIN;
$plugin_description = gettext('Reverse Geocodes images with GPS data.<br /> Adding new images or refreshing the metadata will populate the location, city, state & country fields in the database.
Uses Google\'s Geocode API to convert coordinates to address components, looping over your active languages.

The cURL library is needed to transfer the data.
Requires PHP +5.2 to parse the JSON responses.

');
$plugin_author = "Anton Puttemans (tunafish)";
$plugin_version = '0.1';
$option_interface = 'google_reverse_geocode_options';

zp_register_filter('image_metadata', 'google_reverse_geocode_new_image');

class google_reverse_geocode_options {

function google_reverse_geocode_options() {
setOptionDefault('img_location', 'route');
setOptionDefault('img_city', 'locality');
setOptionDefault('img_state', 'administrative_area_level_1');
setOptionDefault('img_country', 'country');
}

function handleOption($option, $currentValue) {}

function getOptionsSupported() {
return array(
gettext('location') => array(
'key' => 'img_location',
'type' => OPTION_TYPE_TEXTBOX,
'order' => 1,
'desc' => gettext('suggested address component type') . ': route'
),
gettext('city') => array(
'key' => 'img_city',
'type' => OPTION_TYPE_TEXTBOX,
'order' => 2,
'desc' => gettext('suggested address component type') . ': locality'
),
gettext('state') => array(
'key' => 'img_state',
'type' => OPTION_TYPE_TEXTBOX,
'order' => 3,
'desc' => gettext('suggested address component type') . ': administrative_area_level_1 || administrative_area_level_2'
),
gettext('country') => array(
'key' => 'img_country',
'type' => OPTION_TYPE_TEXTBOX,
'order' => 4,
'desc' => gettext('suggested address component type') . ': country'
),
gettext('

Address Component Types documentation on Google Geocoding API

') => array(
'key' => 'note',
'type' => OPTION_TYPE_CUSTOM,
'order' => 5,
'desc' => ''
)
);
}

}

function google_reverse_geocode_new_image($image) {

$lat = $image->get('EXIFGPSLatitude');
$lng = $image->get('EXIFGPSLongitude');
$lat_ref = $image->get('EXIFGPSLatitudeRef');
$lng_ref = $image->get('EXIFGPSLongitudeRef');

// is GPS data found?
if (!empty($lat) && !empty($lng) && !empty($lat_ref) && !empty($lng_ref)) {

// convert N-E-S-W to decimal
($lat_ref == "N") ? $lat_ref = "" : $lat_ref = "-";
($lng_ref == "E") ? $lng_ref = "" : $lng_ref = "-";
// string for Google
$latlng = "$lat_ref$lat,$lng_ref$lng";

// multilanguage arrays for the fields
$arr_location = $arr_city = $arr_state = $arr_country = array();

$_languages = generateLanguageList();
foreach ($_languages as $text => $locale) {

$locale_code = substr($locale, 0, 2);
$geocodeURL = "http://maps.googleapis.com/maps/api/geocode/json?latlng=$latlng&language=$locale_code&sensor=false";

// Curl them (not tested on performance and delays yet)
$ch = curl_init($geocodeURL);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$result = curl_exec($ch);
$httpCode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
curl_close($ch);

// continue only if we get the correct response
if ($httpCode == 200) {
$data = json_decode($result);

if ($data->status == "OK") {
if (count($data->results)) {
foreach ($data->results[0]->address_components as $component) {
if (in_array(getOption('img_location'), $component->types)) $arr_location[$locale] = $component->long_name;
if (in_array(getOption('img_city'), $component->types)) $arr_city[$locale] = $component->long_name;
if (in_array(getOption('img_state'), $component->types)) $arr_state[$locale] = $component->long_name;
if (in_array(getOption('img_country'), $component->types)) $arr_country[$locale] = $component->long_name;
}
}
}
}
}

// save to the DB
if (!empty($arr_location)) $image->setLocation(serialize($arr_location));
if (!empty($arr_city)) $image->setCity(serialize($arr_city));
if (!empty($arr_state)) $image->setState(serialize($arr_state));
if (!empty($arr_country)) $image->setCountry(serialize($arr_country));

$image->save();
}

return $image;
}

?>
`

sbillard

I also need to test what happens on an installation with the multi language not active btw.

If you correctly handle multi-lingual fields there should be no issue with sites that are not multi-lingual. At least not from the Zenphoto standpoint. (BTW: I did not review your code. Too much trouble here. Maybe you can create a ticket and attach it there. That would be much more convenient to browse.)

I have not tested it against the xmpMetadata plugin. I think you can set some sort of priority right? Is this necessary or will one just override the fields in the database?

Filters can be prioritized. As things stand, they will apply their changes in the order they run, so the "last" one wins. The filters all happen after the normal within the image metadata is stored. If there are multiple plugins handling metadata it is an interesting question as to which one should take precidence. As well at present there is no way for the site manager to change the priorities short of hacking the code. Maybe something we should consider.

What I would also like to integrate is some sort of lock if the location, city, state & country fields are already filled out. For example I reverse_geocode the image(s) and than correct the location field with admin-edit.php. The next time I do a "refresh metadata" the fields will be overriden.. I would like some option to lock a particular field from a particular image. I think it's especially useful for the location field.
For example, instead of the geocoded street name I could want something like "old town".

Your plugin can test to see if the field is "empty" and only store if it is. Of course if you do this, then even if the data changes your plugin will not update it. (Not sure how the geo location could change, but I suppose there could be changes in the political boundries.) Refresh Metadata is really intended to do just what you wish to avoid.

sbillard

Update: I notice that you use CURL. Please note that this feature is often not enabled. The `tweet_news` plugin also needs this feature. Take a look at its '$plugin_disable` line to see how we disable plugins when features they need are not present.

tunafish

Ok I could use `file_get_contents` if cURL is not enabled.
Good point about only populating the field when it is empty. That makes sense because this data can be edited from the admin anyway. So I will slipstream the "empty" statement.
I will update the plugin and do more tests in a few days as I am occupied this week.

Laters.

acrylian

Note that our metadata refresh will generally clear fields if I am not mistaken, even if edited afterwards!

sbillard

Without reading the script I have a question on using `file_get_contents()` If that has a URL as a "file path" that also may be disabled on a site. Anyway, I suggest you just require `cURL` to run the plugin.

tunafish

OK, I think I got it now.

I took into account all your considerations and added a "rescan" option that will give control on preserving edits.
By default the "rescan all" option is unchecked and only images that are not geocoded (no data in the fields) will process. You can actually rescan specific images and individual fields when you clear a field in admin-edit and to a "metadata refresh"
When the option is checked all images and fields can be re-geocoded again, overriding the old values. This option could also be refined by specifying which fields should be scanned for all images.
I think these options give you a lot of control. (at least if understood, I hope I did the descriptions okay)

I tested both single and multi language installs and implemented the `$plugin_disable` if `curl_init` does not exist.

The plugin will not work when `IPTC sublocation`, `IPTC city`, `IPTC state/province` or `IPTC country` data is present in the image (for example you could do this in Lightroom)
ZP is too dominant in this case :-P

`
<?php

/**
* @author Anton Puttemans (tunafish)
* @package plugins
*/

$plugin_is_filter = 9|CLASS_PLUGIN;
$plugin_description = gettext('Reverse Geocodes images with GPS data.<br />
Populates the location, city, state and country fields in the database.

When adding new images or hitting the "refresh metadata" button in the overview tab Google\'s Geocode API is queried to reverse geocode the GPS data.
The plugin will convert coordinates to address components, looping over your active languages.


Note: This plugin will not work when the image has IPTC sublocation, IPTC city, IPTC state/province or IPTC country metadata.
In this case ZP will automatically populate these fields for you, but NOT multilanguage..

The cURL library is needed to transfer the data.
PHP +5.2 is required to parse the JSON responses.

'
);
$plugin_author = "Anton Puttemans (tunafish)";
$plugin_version = '0.2';

$plugin_disable = (function_exists('curl_init')) ? false : gettext('The php_curl extension is required');
if ($plugin_disable) {
setOption('zp_plugin_google_reverse_geocode', 0);
} else {
$option_interface = 'google_reverse_geocode_options';
zp_register_filter('image_metadata', 'google_reverse_geocode_new_image');
}

$option_interface = 'google_reverse_geocode_options';

class google_reverse_geocode_options {

function google_reverse_geocode_options() {
setOptionDefault('img_location', 'route');
setOptionDefault('img_city', 'locality');
setOptionDefault('img_state', 'administrative_area_level_1');
setOptionDefault('img_country', 'country');

setOptionDefault('rescan', 0);
}

function handleOption($option, $currentValue) {}

function getOptionsSupported() {
return array(
array(
'type' => OPTION_TYPE_NOTE,
'order' => 0.5,
'desc' => "

" . gettext("Note: leave fields empty if you don't want them to process.") . "

"
),
gettext('location') => array(
'key' => 'img_location',
'type' => OPTION_TYPE_TEXTBOX,
'order' => 1,
'desc' => gettext('suggested address component type') . ': route'
),
gettext('city') => array(
'key' => 'img_city',
'type' => OPTION_TYPE_TEXTBOX,
'order' => 2,
'desc' => gettext('suggested address component type') . ': locality'
),
gettext('state') => array(
'key' => 'img_state',
'type' => OPTION_TYPE_TEXTBOX,
'order' => 3,
'desc' => gettext('suggested address component type') . ': administrative_area_level_1 || administrative_area_level_2'
),
gettext('country') => array(
'key' => 'img_country',
'type' => OPTION_TYPE_TEXTBOX,
'order' => 4,
'desc' => gettext('suggested address component type') . ': country'
),
gettext('rescan all') => array(
'key' => 'rescan',
'type' => OPTION_TYPE_CHECKBOX,
'order' => 5,
'desc' => gettext('If unchecked, only images with no data in the fields location, city, state or country will be reverse geocoded.

Tip: You can rescan individual fields for specific images by clearing the data in admin-edit images


If checked, all images will be rescanned and reverse geocoded again.

Warning: Rescanning will override all fields again so any individual field edits will be lost.'
)
),
gettext('

Address Component Types documentation on Google Geocoding API

') => array(
'key' => 'note',
'type' => OPTION_TYPE_CUSTOM,
'order' => 6,
'desc' => ''
)
);
}

}

function google_reverse_geocode_new_image($image) {

$lat = $image->get('EXIFGPSLatitude');
$lng = $image->get('EXIFGPSLongitude');
$lat_ref = $image->get('EXIFGPSLatitudeRef');
$lng_ref = $image->get('EXIFGPSLongitudeRef');

$db_location = $image->get('location');
$db_city = $image->get('city');
$db_state = $image->get('state');
$db_country = $image->get('country');

// is GPS data found?
if (!empty($lat) && !empty($lng) && !empty($lat_ref) && !empty($lng_ref)) {

// convert N-E-S-W to decimal
($lat_ref == "N") ? $lat_ref = "" : $lat_ref = "-";
($lng_ref == "E") ? $lng_ref = "" : $lng_ref = "-";
// string for Google
$latlng = "$lat_ref$lat,$lng_ref$lng";

// MULTI LANGUAGE
if (getOption('multi_lingual')) {
// multilanguage arrays for the fields
$arr_location = $arr_city = $arr_state = $arr_country = array();

$_languages = generateLanguageList();
foreach ($_languages as $text => $locale) {

$locale_code = substr($locale, 0, 2);
$geocodeURL = "http://maps.googleapis.com/maps/api/geocode/json?latlng=$latlng&language=$locale_code&sensor=false";
$ch = curl_init($geocodeURL);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$result = curl_exec($ch);
$httpCode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
curl_close($ch);

if ($httpCode == 200) {
$data = json_decode($result);
if ($data->status == "OK") {
if (count($data->results)) {
foreach ($data->results[0]->address_components as $component) {

if (getOption('rescan')) {
if (in_array(getOption('img_location'), $component->types)) $arr_location[$locale] = $component->long_name;
} else {
if (empty($db_location)) {
if (in_array(getOption('img_location'), $component->types)) $arr_location[$locale] = $component->long_name;
}
}

if (getOption('rescan')) {
if (in_array(getOption('img_city'), $component->types)) $arr_city[$locale] = $component->long_name;
} else {
if (empty($db_city)) {
if (in_array(getOption('img_city'), $component->types)) $arr_city[$locale] = $component->long_name;
}
}

if (getOption('rescan')) {
if (in_array(getOption('img_state'), $component->types)) $arr_state[$locale] = $component->long_name;
} else {
if (empty($db_state)) {
if (in_array(getOption('img_state'), $component->types)) $arr_state[$locale] = $component->long_name;
}
}

if (getOption('rescan')) {
if (in_array(getOption('img_country'), $component->types)) $arr_country[$locale] = $component->long_name;
} else {
if (empty($db_country)) {
if (in_array(getOption('img_country'), $component->types)) $arr_country[$locale] = $component->long_name;
}
}

}
}

if (!empty($arr_location)) $image->setLocation(serialize($arr_location));
if (!empty($arr_city)) $image->setCity(serialize($arr_city));
if (!empty($arr_state)) $image->setState(serialize($arr_state));
if (!empty($arr_country)) $image->setCountry(serialize($arr_country));

$image->save();
}
}
}

}

// SINGLE LANGUAGE
else {
// single language strings for the fields
$str_location = $str_city = $str_state = $str_country = "";

$locale = getUserLocale();

$locale_code = substr($locale, 0, 2);
$geocodeURL = "http://maps.googleapis.com/maps/api/geocode/json?latlng=$latlng&language=$locale_code&sensor=false";
$ch = curl_init($geocodeURL);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$result = curl_exec($ch);
$httpCode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
curl_close($ch);

if ($httpCode == 200) {
$data = json_decode($result);
if ($data->status == "OK") {
if (count($data->results)) {
foreach ($data->results[0]->address_components as $component) {

if (getOption('rescan')) {
if (in_array(getOption('img_location'), $component->types)) $str_location = $component->long_name;
} else {
if (empty($db_location)) {
if (in_array(getOption('img_location'), $component->types)) $str_location = $component->long_name;
}
}

if (getOption('rescan')) {
if (in_array(getOption('img_city'), $component->types)) $str_city = $component->long_name;
} else {
if (empty($db_city)) {
if (in_array(getOption('img_city'), $component->types)) $str_city = $component->long_name;
}
}

if (getOption('rescan')) {
if (in_array(getOption('img_state'), $component->types)) $str_state = $component->long_name;
} else {
if (empty($db_state)) {
if (in_array(getOption('img_state'), $component->types)) $str_state = $component->long_name;
}
}

if (getOption('rescan')) {
if (in_array(getOption('img_country'), $component->types)) $str_country = $component->long_name;
} else {
if (empty($db_country)) {
if (in_array(getOption('img_country'), $component->types)) $str_country = $component->long_name;
}
}

}
}

if (!empty($str_location)) $image->setLocation($str_location);
if (!empty($str_city)) $image->setCity($str_city);
if (!empty($str_state)) $image->setState($str_state);
if (!empty($str_country)) $image->setCountry($str_country);

$image->save();
}
}

}

}

return $image;
}

?>
`

tunafish

I have put it up on: http://code.google.com/p/zenphoto-google-reverse-geocode/

acrylian

That is better as the code above is unreadable..;-) Is it already ready and working? Then I would add an entry on the extensions section.

tunafish

What do you mean unreadable?
I used the backticks around the code. A lot of tabs, you just need to scroll?
Anyway yes it is tested and working. At least on my localhost installation it is...
One thing I need to do is the translations. I just read about the gettext_pl() this morning...
I will update it soon to v0.3

Not sure what you mean about an entry in the extensions section but I will check it out.
Doing this plugin was a great experience btw. The more time I spend with ZP the more I realize how awesome ZP is! If I would have only known a earlier :-)
Big thumbs up to the ZP team for all knowlage and work you guys have put into this, but most of all the support on this forum.

Soon when things calm down at work I will put my site online. And as soon as it is make a donation!
Cheers!

acrylian

Well, not unreadable in the true sense, it just does not make any fun to look at such long code on the forum..;-)

Not sure what you mean about an entry in the extensions section but I will check it out.

www.zenphoto.org/news/category/extensions
There you find entries of all official and also all third party extensions (at least all we know of).
You cannot add it yourself so we do and link to your page where you host your plugin. That way other find it easier in case they need it. (see the "Get involved" link on top or directly the "Contributor guidelines"). In this case tha would be probably the google code page (but later maybe your own site as you surely would benefit from our sites rank and traffic).