Since you installed Python 3.11.3 from Python.org, its binaries are located linked into /usr/local/bin. You can completely ignore the (optional) Command Line Tools for Xcode (or Xcode itself) installation of Python3 and pip3 in /usr/bin, and these cannot be removed as they are on a read-only System volume.
In your Shell startup files, either ~/.zshrc, or ~/.bash_profile, change your PATH statement:
export PATH=".:/usr/local/bin:/Library/Frameworks/Python.framework/Versions/3.11/bin:${PATH}"
From the Terminal, after the above PATH change, you can run the following to change the PATH in the current Terminal window where dotfile is either zshrc, bashrc, or bash_profile:
source ~/.dotfile
Now, when you run python3 or pip3, they completely ignore the older Apple python3 or pip3 and all third-party packages get installed in the correct site-packages location in that python.org distribution. If you installed any third-party packages using the Apple Python binaries, remove those packages, and after the above changes, reinstall those packages into the python3.11.3 distribution's site-packages using its pip3.
I go one step further and use python to create a virtual environment (venv) to isolate third-party packages to one filesystem location.
mkdir -p ~/py_projects
python3 -m venv ~/py_projects/env311
source ~/py_projects/env311/bin/activate && cd ~/py_projects/env311
pip3 install -U pip
pip3 install -U jupyterlab
pip3 install -U notebook
︙
deactivate # when you are done with the venv
cd - # back to the previous directory location
The Python3 version in this venv is updated when you apply any future Python 3.* updates from Python.org.