You won't be able to import the library without restoring it to your boot drive with Time Machine.
To see if you can restore the library from the old Time Machine drive hold down the option key (with the old TM drive connected to your iMac and select "Browse Other Backup Discs¬" from the TM menu.

The locate and restore the library. You may have to select "Keep Both" from the menu that pops up if the restored library has the same name as your current library:

You may not want to use Photos to migrate the restored library into your current library if you have a lot or organizational effort in the old library. You may want to use the paid version of PowerPhotos. Here is a comparison between PowerPhotos and Photos:
I ran a number of tests with a 3119 photo, 72 video and 97 keyword Photos library with keywords, captions, keywords and locations. I merged it into an empty library using Photos and PowerPhotos. This is what I found was imported by each;
PowerPhotos: Photos:
Albums*** -----------
Smart Albums (as regular albums)** -----------
Captions Captions
Titles Titles
Keywords* Keywords (some but not all in tests)
Original images Original images
Edited images Edited images
Loctions Locations
Favorites Favorites
Can detect and exclude Duplicates
duplicates upon import or not
Neither method could import/merge projects.
Both methods could import older iPhoto libraries into a Photos library when the iPhoto library couldn't be migrated into a Photos library.
*The Photos app only imported 84 keywords our of 97. PowerPhotos merged 147 keywords, some which apparently were in the images but not read by Photos at the time of import or keywords delete along with photos but not purged from the database.
**The original library had 90 Smart Albums. All were merged by PowerPhotos but, as indicated above, the Smart albums were brought over as regular albums. No album of any kind were imported by Photos.
***The original library had 34 regular albums and 1 folder with some nested albums. PowerPhotos merger all of them successfully.
The original library had 72 videos. Photos imported only 57. PowerPhoto got all 72 in its merge.
