Duplicate photos from data transfer not detected by Mac's utility scan

Some background information - There are two different scans for duplicates in Photos for Mac:

• When we import photos to Photos it is scanning for exact duplicates - identical files. This is based on the file attributes. The filenames may differ, but the contents of image file and the file format and the resolution needs to be the same. when we try to import the same image again, we will get a warning, but are allowed to import the duplicate, when we insist. The duplicate detection on import is very efficient. There are no exact duplicates in my library, unless I duplicated o photo to use it twice in a slideshow. Duplicates we create intentionally in Photos do not need extra storage.
• The Duplicates Collection in Photos will show us not only identical items but also similar items, based on the contents of the image file. Photos will start the duplicates detection only after it has scanned the library and understood whet the photos are showing. For my current iCloud Photos library with approximately 80000 photos and videos it took three weeks until the first duplicates have been shown. The Photos.app is doing the scan in the background, when we are not using our Mac, because it needs a lot of processing power. Over time, more and more duplicates will appear in the album.

I am using the Duplicates album just as a hint, which items to check for duplicates, but not to delete the duplicates en masse. They are not exact duplicates and I want to be the judge which version to keep, for different reasons.

• Different Metadata: The album is also including items, where the dates, locations, the titles, captions keywords or adjustments are different. Photos cannot know, which metadata are the correct ones and should be preserved. So I have to check each pair of apparent duplicate manually and consolidate the metadata before deleting one of them.
• The apparent duplicates may also be differently edited versions: These are duplicates, where I need to keep all versions, because I created them intentionally to have differently cropped versions, or a black-and-white version in addition to the color version, or Live Photos with different animation effects.
• Similar photos taken in a quick succession: Sometimes Photos is pointing out similar looking photos as duplicates, for example action shots taken in quick succession, or group photos. I rarely like the photos that have chosen as the keeper by Photos and Peter to pick the best photos myself.
• Items with a different quality and resolution: Photos is selecting the version with the best resolution (width and height), but the least file size, if the resolution is the same. I prefer to keep the version with the least compression, if I have not yet done with editing.

I'm not sure how you doubled up on all those pictures. Photos scans for duplicates during the import process.

How long have you waited? It takes a really long time for Photos to scan every picture and then compare results of each on to every other one.Seriously, if you had 1000 pennies, how long would it take to find all duplicate dates? A computer may be faster, but there are a whole lot more differences to be found in pictures.

If the duplicates were added at a different time, you may be able to look for Recently Saved, or you could use a Smart Album to separate them out. There are a couple of 3rd party apps that we trust, but there are a lot out there that will mess up the Photos Library. But you should wait.

The two apps that we trust are the very versatile  PowerPhotos ($35,) and  PhotoSweeper ($15) which is cheaper but does less.

léonie wrote: Some background information - There are two different scans for duplicates in Photos for Mac:

That is a great summary of Duplicates! Thanks! It can be very confusing that "Duplicate" doesn't always mean the same thing!

I wonder if SylviaRWarren's duplicates are actually different in some way-- maybe the new ones are jpeg and the older ones are heic, for instance, so the different formats were allowed through. If she can identify the difference, then she could use a Smart Album like this:

to find the HEICs (or JPEGs or JPGs ) imported in the last few days. Or she might hunt for duplicates not in an album, or which are missing GPS data, etc.

The smart album is a good idea. It is pity that Sylvia cannot tell us exactly, how the photos have been transferred to the new Mac. If the duplicates differ in a systematic way, it would be easier to track them down.

If the Apple Store made a mistake, it might be easier to repeat the migration of the photos and just copy the old Photos Libraries over from the old Mac to the new one.

SylviaRWarren wrote: How long would you estimate the "Finding Duplicates" utility takes for 4k photos? A few hours?

Maybe two or three days, with Mac on and not being used—left on overnight is good.

It also helps if your don't allow the Mac to go to sleep.