Delayed Apple Home responsiveness for non-owner users

Hello,

I am having this issue where an Apple Home is constantly taking at least 2 minutes to become "responsive" after opening the Apple Home App for all people that are joined to the Home EXCEPT the Owner of the Home connects immediately - The Apple Home "Owner" has no "Homehub is not respondin" error message

It has been an issue as the fact it takes 2 Minutes to become "responsive" means that when hey go to their Level Lock and try to use their Apple HomeKey to Open the door it just doesn't work....

The Owner has an "optusnet.com.au" iCloud if that makes any difference

Currently their is only an Apple HomePod mini as the Home Hub next to the Door and also next to the Modem for a strong WIFI Connection

I tried using an Apple TV 4K with Ethernet as na additional Home Hub. but this did not make any difference and the same issues arised...

I have setup about 50+ Apple Homes for People and this is the only time I have seen it on such a small setup... has anyone experienced this before?

Thanks!

[Re-Titled by Moderator]

Original Title: Apple Home - No Response - Only to Other Admins - Not Owner of Account