Is it possible that the Audio files BPM differ from that of the Project Tempo?
This thread might help you understand my angle here:
Logic Pro X playback pitch and timing mes… - Apple Community
I just did a basic test where my project bpm is 90 and my Audio Region has a bpm of of 146.
When I turn on Flex it manipulates the audio in an undesirable way as the Audio Project and Region BPM are not in sync. The Project Tempo and Region Tempo have to be aligned for things to work smoothly.

Open up the Audio "Smart Tempo" Editor and Analyze the audio and this should give you more data to solve this issue friend. After that, you can use the "Apply Project Tempo to Region and Downbeat". Best of luck!
