How to prevent MacBook's cursor from connecting to iPad in Universal Control?

So i have have a Macbook Pro as my main driver, and a a mac mini connected to the TV and then I also have an iPad which I use to play music & other stuff. I have universal control turned on on for the Mac's, but I don't want it constantly connecting to my iPad, the courser keeps moving over to the iPad & it's super annoying - is there any work around for this?

I've already turned off airplay, handoff and all those settings but it's still connecting which is super frustrating.

[Re-Titled by Moderator]

Original Title: AirPlay & Handoff turned off on my iPad but my Macbook courser still connects to the iPad? How to stop without turning off universal control?