How do I delete an album.

Control-click (or right-click) on the album icon or name in the sidebar to bring up the context menu.

Select "Delete Album" from that context menu.

The photos will stay in the library (and any other albums in which they might appear).