Problem with turingtest2 DeDuper Script

Question

Level 1

4 points

Problem with turingtest2 DeDuper Script

The post is for turingtest2. Thank you for putting an incredible amount of time to write all of your scripts.

My name is Greg. I am an IT support tech helping a friend with a problem with his iTunes library. I am using my friends account to post this message.

His library is huge. Hundreds of thousand songs. I discovered a massive amount of logical or pseudo duplicates (i.e. Library entries that point to a single music file). There were physical and alternate duplicates as well. DeDuper to the rescue!

I have cleaned up all the duplicates but encountered a problem early on in the process. The Script crashed. My investigation showed it was crashing when it encountered a library entry that was pointing to a missing song. I removed all of the missing song entry's from the library. This allowed me to complete the cleanup.

To reproduce the problem you must have at least one duplicate song and one of the duplicate entry's to be a missing file. Here is a screen shot of the error...

Windows, Windows 6

Posted on Feb 13, 2025 09:00 AM

Reply

Answer 1

turingtest2

Level 10

277,723 points

Feb 13, 2025 09:42 AM in response to TomTunes

Hi Greg,

Thanks for the feedback. I'm busy over the next few days but I will try to find time to replicate this and trap/workaround the error. The script should normally handle the case where one duplicate exists at its given path, while another returns an empty location, as this allows for ratings, plays, and playlist membership to be concentrated on the track that is preserved. Do you happen to know if this was during the initial scanning phase, or the second clean up phase?

tt2

Reply

Answer 2

TomTunes Author

Level 1

4 points

Feb 13, 2025 11:02 AM in response to turingtest2

Hi,

Thanks for the speedy reply! I am not sure but I think it was during the scan phase. I think during the 3rd phase. But do not hold me to it.

I just looked at the script. the error on line 746 is contained in Subroutine Merge your comment for that routine is "Merge metadata for two tracks, choosing which to preserve and prepare to delete the other". The failure occurs within Case 2 with comment "Physical: Check for pseudo-physicals first".

Not sure if this is relevant but I think the first instance of this in the library had 4 entries for the song. Three were pseudo duplicates. Only one out of the three pseudo was missing.

Honestly, I am not sure how the **** this library got so messed up. I kept track of the numbers. WOW!

Before DeDuper

______________

All - 268629

Missing - 11604

After Deduper

______________

All - 139492

Reply

Answer 3

TomTunes Author

Level 1

4 points

Feb 13, 2025 11:53 AM in response to TomTunes

Hi,

I decided to load up the original library itl that I saved to help refresh my memory. I think I found the first culprit. I got it wrong in the last post I made. Please ignore that.

The song is from the same artist, album. It has 4 library entries. Here is the breakdown...

1 - legit entry

1 - pseudo

2 - missing

Hope this helps

Reply

Answer 4

turingtest2

Level 10

277,723 points

Mar 17, 2025 11:42 AM in response to TomTunes

Apologies for this oversight. I've made some amendments to the script that should handle this error if it occurs again and lists some information about the tracks involved. I've tested a small selection of pseudo-physical and actual physical duplicates here cleaning 32 tracks down to 8 originals as expected. In principle that error shouldn't occur as both files that are being compared have had an existence check before you get to the line that caused the error, and it shouldn't error if there is a file at that path. But hey, if everything worked as we had expected there would never be any bugs.

tt2

Reply