canary mismatch on efree() - heap overflow detected (functions-basic.php KSES)

n20capri

Errored again...That didn't take long. At least this time my css loads - ZERO content or images other than the background layout...weird. Restarting apache brought it back as usual.

Exact error:
[Wed Jul 11 17:27:35 2012] [error] [client xx.xxx.xxx.xxx] ALERT - canary mismatch on efree() - heap overflow detected (attacker 'xx.xxx.xxx.xxx', file '.......zp-core/functions-basic.php', line 877), referer: http://www.mikemartinelli.com/colorbox.css

What's the deal with colorbox this time? I know there was a path change with the new version of ZP but I couldn't get colorbox to work through ZP so I changed it up a bit using the path above...worked before and works fine now...

ZP version 1.4.3 [10393]

I'm also seeing some new debug-log entries that I didn't see before...

{Tue, 10 Jul 2012 23:10:39 GMT} Zenphoto v1.4.3[10393]

Backtrace: WARNING: Cannot modify header information - headers already sent by (output started at ......zp-core/zp-extensions/zenpage/zenpage-admin-functions.php:194) in ...........zp-core/admin-functions.php on line 109 header called from printAdminHeader (admin-functions.php [109]) from admin-edit.php [120]

Also this one:
{Wed, 11 Jul 2012 02:07:08 GMT}

Backtrace: NOTICE: Undefined index: HTTP_USER_AGENT in .......zp-core/zp-extensions/hitcounter.php on line 158

hitcounter::load_script called from call_user_func_array (unknown) from zp_apply_filter (functions-filter.php [150])
from index.php [71]

sbillard

Here is what I found from a search on the cannary mismatch error:

http://www.suspekt.org/2008/10/12/suhosin-canary-mismatch-on-efree-heap-overflow-detected/

Not at all sure what this means, though.

"cannot modify header information" errors are almost always the result of up-stream errors.

The one with hitcounter seems strange, I do not know why there would not be a user agent specified. But that code is only invoked if the option `hitcounter_ignoreSearchCrawlers_enable` is set for hitcounter.

n20capri

Yeah I've read a lot about the error - didn't get too far with server changes...figured I'd try to figure out what is triggering it on the ZP side...

don't know what would cause the header info or hitcounter errors either...prior to the upgrade I didn't have any debug errors...

sbillard

Since so many people had problems finding their server logs we made Zenphoto log the errors it can. This is new to 1.4.3, so you would have seen these errors only in the PHP error log on prior releases.

The canary-mismatch seems to be either a bug in PHP or in the suhosin patch. Seems some debate as to which. I would presume that if it is a PHP issue it is caused by large scale memory use. Maybe the kses examination of input data creates more strings than PHP can handle. It does sound like there is an IP address in the message. Is that an address you are familiar with?

acrylian

Regarding the user agent: Isn't it possible to hide or not send that via privacy browser extensions? (Just a wild guess)

n20capri

The IP addresses vary - my guess is they are just visitors IP's - I see mine in there when I'm playing around too...I also see the same ones that ZP is logging as well.

What a mess! lol

acrylian

I meant each browser sends a user agent (type of browser or bot etc). For some reason that not being sent or masked it could somehow confuse maybe...as said a very wild guess.. You are really the only one we ever heard having this error.....At least you take if with a bit fun..;-)

n20capri

Well there is nothing else I can do but have fun with it. At least its not a client and just my personal site.

I appreciate the continued help. I'm sure I will be back reporting something. Ha

n20capri

Beating a dead horse I know but talking about things ALWAYS helps!

BUT, I guess the biggest question I have is why isn't this happening with other ZP installs I have? I have a client that gets more traffic than me with regular updates...that is version 1.2.8 and is always super smooth...All on the same server...

It ONLY happens on the www domain of my site...not any other sub-domains...weird...There has to be something specific going on that I can't figure out...

acrylian

I fear we cannot help. The first step is to reproduce the issue and well, we are not able to do so...

n20capri

Gonna bring this back again...lol. Just talking out loud and asking more questions trying to maybe trigger what is causing this...Things were fine for several weeks but lately it's been happening again...

I DO appreciate any input...

The latest 2 errors are coming from functions-basic.php line 882:
$output_string = kses($input_string, $allowed_tags);

Then the other reference is template-functions.php line 4517:
$short2 = kses($short.'</p>', $allowed_tags);

I'm seeing both are associated with kses and allowed_tags. However, my site doesn't allow users to add tags and I only have a few tags assigned.

It also seems to always be associated with colorbox as the referrer.

I know we have yet to be able replicate this error, but I'm just trying to brain storm ideas and figure out what's exactly happening with the code in question.

Thanks for your patience.
Mike

acrylian

Allowed tags doesn't mean "tags". Actually it is the wrong word because it refers to the html elements allowed in fields like comments or other fields that don't use the TinyMCE editor (sometimes html elements are called "tags" as well).

What are the actual errors you got?

sbillard

First, I suspect that there is some confusion of terms here. Tags is unfortuately used in two different contexts here. The first, in "allowed_tags" refers to HTML markup tags. "Tags assigned" presumably refers to image and album tags which are used for searches, etc.

Zenphoto will always parse user input for potential hacker ploys. On of the things it does is purge any HTML tags that are not on the "approved" list. That is what the kses function is doing.

It could well be that some hacker is exploiting your site and passing strings that are somehow causing the kses function to crash. What else besides colorbox is in the referer chain? Maybe that is a clue.

n20capri

Ok good - I had a feeling it was something to do with comments. I changed spam filters a little while back so I'm not getting ANY spam comments coming though and this error has seemed to be happening less frequently since changing spam filters - but I guess they are still trying to spam and for whatever reason maybe it's triggering this error? idk. I suppose I could disable comments all together but only as a last resort.

Full error:
[Mon Sep 24 11:54:20 2012] [error] [client xx.xxx.xxx.xxx] ALERT - canary mismatch on efree() - heap overflow detected (attacker 'xx.xxx.xxx.xxx', file '......../zp-core/functions-basic.php', line 882), referer: http://www.mikemartinelli.com/colorbox.css

Attacker IP address could be anyone visiting the site at that time - not necessarily an attacker.

Sometimes the referrer is colorbox, sometimes it's index.php and sometimes it's an external site I've posted on which links back to my site...sometimes there is NO referrer at all.

Thanks again

sbillard

I am not sure how loading a css can cause this. But what you need is to get a stack trace of the failure if you can. That will tell you what functions are in the loop and perhaps then what is going on.

n20capri

I don't know how to get a stack trace but I will try to figure it out.

Another thing I just noticed...When the referrer is an external link I've been noticing a lot of times it's from a message board thread with images linked from my site that have been moved or no longer exist...from YEARS before I started using ZenPhoto. Not sure if that matters but I'm throwing it out there in case it jogs someones memory.

Thanks

acrylian

Hm, not found images should just result in such errors I would think (404 actually). If those have specific paths from the old site you could probably block those with an 404 using htaccess directly.

n20capri

Do you know if there is a wildcard for 404 htaccess? For example:

404 All images that used to be

http://www.mikemartinelli.com/testweb/newfolder/*

Something like that?

acrylian

Sure, similar wildcards do exist, not exactly with just * though. See the Zenphoto htaccess for some redirection examples or the net.

n20capri

Well same exact error happened to a clients site now! Ughhh!

Same error but no referrer like I get with my site...

His site has been perfectly fine for YEARS w/o issue. I recently (less than 2 months ago) upgraded from an older version to 1.4.3.2.

I had thought my issues might have been related to comments...BUT I have comments completely disabled on his site so I'm now even more baffled!

http://www.davidsonscollisioncenter.com

Not sure if this will help track down the issue but it's something new so I figured I'd post about it.

Thanks,
Mike

acrylian

Really a mystery this one...

sbillard

As previously discussed, this error is occurring when Zenphoto sanitizes any web-client input, so does not necessarily have anything to do with comments.

It is (depending on who you ask) a bug in PHP or in the suhosin patch to PHP. The problem is fairly well discussed, just do a search on canary mismatch and you will see. Certainly this is out of the realm of what Zenphoto can correct.

It probably happens when hackers attempt to compromise you site, but of course that is speculation.

n20capri

That's the weird thing...there aren't any forms on my clients site for user input...not even a contact form. Definitely more baffled now! ha.

sbillard

Would not even have to be a form--maybe just a forged link with a `?album=some strange string` Or they could be trying to forge a post on your admin pages.

Lots of opportunities to try to hack you, but we hope all are prevented--the error is related to one of the prevention screens.

Problem is that you also apparently have the suhosin patch. I do not know exactlly what that patch is supposed to do other than be a security thing, but it has come up more than once in the past as causing particular issues to Zenphoto.

n20capri

Yeah I've read a lot on the suhosi patch but there is no definitive "fix" that I've found. I've tried many things on the php side of things trying to cure the issue as well.

Short of re-installing PHP w/o it which is too scary! lol

sbillard

Yes, it does seem that there is finger pointing going on with this problem. The PHP folks think it is a problem with the patch, the suhosin ones think it is a bug in PHP. Who knows.

I doubt re-installing PHP would help unless maybe a more recent version (if you are not there already.)

acrylian

My live server has also the suoshin extension as do a few sites that I did but I never saw this one.

n20capri

Well I'm up for trying whatever on the server side - if you guys can think of something I'm all ears! lol

acrylian

I fear my colleague mentioned all. It is a server things, nothing we can do about.

n20capri

Well it's been happening a lot lately with my site...I've eliminated the colorbox.css script it was referring to because that was showing as the referrer mostly. I had a separate colorbox script for pages.php pages - perhaps I had something screwy and it wasn't setup properly...

The colorbox for the image pages is still in place as that doesn't seem to be the issue...

Anyway just an update for now I'll continue to keep this thread updated.
Mike