Plugins should not be able to delete anything, but modifying just about everything should be allowed. I am thinking about ordering, rotating/cropping, ect.
The plugins should definitely be theme agnostic and for the most part require only one line or no lines to implement. printRSS() or printComments() and something like ZenVideo would require no changes to themes, but like I said above would require overwriting functions that are built into the core.
Redoing the admin seems like the #1 priority here. After the bones are laid out, all the functionality that albums have can be implemented into subalbums, something that is badly needed. As I see it, the plugin functionality should then follow, IMO. You guys are basically the developers now, I'm just thinking out loud!