In macOS Catalina and later iTunes was replaced by Music. With iTunes the database that recorded what was in your library was typically stored at ~/Music/iTunes/iTunes Library.itl, where ~ is your user's home folder and the .itl file extension might be hidden. With Music the database is normally inside ~/Music/Music/Music Library.musiclibrary, where again the extension might be hidden. This is actually a package, a special type of folder that appears as a single file with a custom icon, when view with Finder. As a user there is normally no need to investigate what is inside it, but while iTunes had some visible support files in the iTunes folder (.itdb, .xml, Album Artwork folder) these are hidden away with Music. The library database is updated each time you do anything with the library. While iTunes could be configured to generate an XML following any change the same is not true of Music so presumably you used File > Library > Export Library to generate the XML you refer to. The XML files can be used by third party software to see what is in your library and where to find it, but iTunes/Music don't normally refer to them. The .itl/.musiclibrary file/package is your library.
Whenever you change a tracks properties in iTunes or Music that data is, where possible, written to the database and the tag of the file where such data can sometimes be shown in Finder. Examples of properties that are not stored in tags, and are only in the database or its support files, are date added, ratings, play & skip counts & dates, playlist membership, associated artwork (downloaded automatically and not explicitly embedded by you), some fields in .aif files, and all metadata associated with .wav files.
If you are working with a library that started life as an iTunes library and has been migrated into Music at some point see Managing your Mac media libraries - Apple Community for an article that explains some of the component parts of the different libraries, how you might massage a working library into a more standard layout for music, and what parts of an old iTunes library can now be discarded. Provided for general background. Don't move things around that are currently broken or it will make matters worse.
In general the library expects that any items that have been added to it stay in exactly the same place once added. Macs can be a little more forgiving in this respect than Windows machines but it still applies. If your library and media are on different volumes that changing the name can break the connection. While it is possible to add media from a manually organized folder structure spread across multiple volumes, keeping to the standard layout, with everything in one drive, makes moving the library to a new drive or computer much easier.
What follows is a boilerplate post I have on missing content in the library and how you might go about fixing it...