In the 'back end' it is possible to choose the order in which images are displayed. e.g. by file name. However, when viewing the album the images are displayed randomly. How can I have my images displayed by file name (or in any other order I may want)?

There are separate settings for the backend and the front end. The front end settings are located on the options or on each album to override the general options.

Thanks, It seems that, for some reason, the sub albums were not following their parent album settings and so I am setting each album/sub album individually to avoid further problems.

There is a separate option on each album for its subalbums. That of course only will work if the subalbums' option is set to use the parent setting. If that is not working it might be a bug. Or - as often - some browser cache issues. Or the theme used doing some custom stuff overriding these options.