I can make a list of albums but I'm confused about how to list the sub albums underneath each album.

I'm basically trying to create a menu tree but I couldn't find any examples of someone doing this.

The print_album_menu does exactly this if that is what you would like to do.

If you mean within the layout you will have to do some PHP coding using the object model. This is of course already advanced coding.