Sorry, that neither of those helped. At this point, I admit that I'm a bit stumpted. So far the only common denominator is Safari running on this Mac ... unless I misunderstood anything in your posts.
Unfortunately, I do not have a Mac available that is running macOS Monterey "natively" to test this out for you. So far, my M2 Pro Mac mini is working "perfectly" with either Safari or Brave (the only web browsers that I use). No real performance difference between them to speak of.
Per chance are you running any anti-virus apps or a software firewall? Also, not knowing anything about your network hardware, it would be hard to assist you with trying to resolve this.
At this point, you can either wait for others to chime in with options you can try or contact Apple Support directly for assistance with this. For the latter, you can simply do so by selecting the "Support" link on the lower left-hand side of this forum's web page. Good luck!