KVM can't switch back to new new M2 Mac Mini

I have a new M2 Mac Mini and a 14-inch M1 Pro MacBook Pro and I'm trying to have them share a 4K monitor (Asus ProArt 279CV) and my USB keyboard/mouse. I bought an ioGear 2-Port 4K USB-C KVM Switch, Model GCS72CC, because it seemed an elegant solution: USB-C cables to each computer, a DisplayPort cable to the monitor, and two USB ports for the keyboard, etc, and a remote switching button.

When I set it up initially, it works just fine: the Mac Mini video connects to the monitor and the keyboard/mouse connect to the Mac Mini. When I press the switch button, the MacBook Pro takes over and it works fine as well. If I switch back to the Mac Mini within a few minutes, that's fine as well, but the problem is this: if it's been switched to the MacBook Pro for a longer time, I haven't been able to switch back to the Mac Mini.

I push the switch button, the KVM indicates it's back to the Mac Mini, but the screen is dark and the monitor complains that there is no signal. If I connect a HDMI cable between the Mac Mini and the Display and switch the input on the monitor, and plug the keyboard directly into the Mac Mini, I can see that it's still awake. But the Thunderbolt ports on the Mac Mini then seem asleep: I can't use keyboard through the KVM or through a USB-C hub. If I restart the Mac Mini, everything works again, until the KVM is switched to the MacBook Pro for too long and this starts again.

I'm perplexed how this can work with the MacBook Pro every time but not the Mac Mini.

Is there any setting I could change on the Mac Mini to fix this problem? Or if not, is there a different KVM setup that is known to work in this situation?