KVM Switch not working on Ventura, Apple M2 Max

I've been using KVM switches to use the same peripherals between my PC and my laptops. Everything worked perfectly fine with my Intel MacBook Pro. However, since I bought the new M2 Max MacBook Pro, the peripherals are no longer detected by the macbook. The monitors work fine, though. I've tried plug-unplug, rebooting, but peripherals just won't work. Plugged back to my old Intel MacBook Pro, and everything was OK.

I've searched online and see people having similar issues, but haven't come up with a work-around yet.

OS: Ventura 13.5.1

Model: Macbook Pro 16