Things that come to mind quickly:
1. set all albums to a fixed album thumb. With that many images it saves you from going through all to find the latest or else.
2. disable the visual thumb selector option.
3. don't upload images bigger than you need for web display (Dimensions, NOT file size). If you need full size images for download maybe better store them separatly.
4. disable the tree view on the backend for hte albums so you only get one level at a time and organize your albums properly in subalbums structure. Zenphoto is generally file system based thus the ease of ftp uploading.
Beside that, I am not sure that any php.ini setting might solve it. You could however try to raise the php memory limit a little more (the more memory the better as always).
PRobably my collegue Sbillard will have some more idea later today.