Mac crashing while on SMB

I have a 2021 M1 Max MacBook Pro running Ventura 13.3.1(a). We are mainly a Windows environment and our Macs are binded to our network. While in a SMB share server, we have many users who experience crashing, freezing, beach balling, and forces a force restart. We use Cisco AnyConnect for VPN and the issues don't happen often when users are remote and connected to the VPN. In office is where we mainly have these issues and aren't able to pin point the cause. Usually when it happens, the users are connected in office on either Ethernet of Wi-Fi, in the shared drive, and have an Adobe or Microsoft program open.

Also some users will attempt to connect to the SMB while connected to ethernet and they'll just get a connecting bar continuously until you quit. When I disconnect them from ethernet, it'll imminently connect to the drive. Some users are opposite in which they can't connect on Wi-Fi but when I connect them to ethernet it'll allow access to the drive.

Troubleshooting steps i've taken:

1) Disconnecting from Wi-Fi or ethernet

2) Unplugging Docking station

3) Unplugging Wacom display

4) Updating computers

4) Followed the document https://forums.macrumors.com/threads/what-is-wrong-with-smb-in-ventura.2386909/

Not sure if it's a Ventura issue but we did have similar issues on Monterey (Just not as often)