How can I stop anyone else from using one of our HomePods?

unlike siri on ipads and iphones which focus on 1 users voice devices such as homepod and appletv focus on all voices not a select group as far as I know

you can try to suggest they make changes using their feedback channels

Product Feedback - Apple

Anyone can use HomePods for general tasks such as playing music, it is only your personal information that is protected by your voice. While you are on vacation to prevent access to your outside HomePods, of course you can unplug it or set up your Router to be able to control which devices are connected to your WiFi network. Some routers will allow you to manage this remotely too.

If it's of great concern, move the HomePod to someplace where you can easily unplug it when not in use.

I was just looking at the settings for my HomePod. Go to "Listen for "Siri" or "Hey Siri". Turn off "Listen On" for the Home Pod. See if that works.

Rudegar wrote:

unlike siri on ipads and iphones which focus on 1 users voice devices such as homepod and appletv focus on all voices not a select group as far as I know

Actually, the reverse is true. Only HomePods can recognize individual voices. iPhones and iPads cannot. However, I don't think you can prevent other voices from accessing basic functions.

Richard Morris wrote:

I was just pondering whether I could prevent it happening whether by way of a prank or by mistake: last evening we were on the patio having dinner and playing some music and our neighbours were also closer than usual to the boundary entertaining. It would only take one of them to try to play some music on their phone for our HomePod to pick up the command instead.

Unless your neighbors consisted of teenagers (or the intellectual equivalent), it seems unlikely that they would do this. And, even if they did, what is the worst that could happen? They ask for country music but you're a classic rock fan?

Take therefore no thought for the morrow: for the morrow shall take thought for the things of itself. Sufficient unto the day is the evil thereof. - Matt 6:34

For the life of me, I can't figure out why your neighbors would do that. Are you involved in some ongoing feud with them?

Take it inside or unplug it when you're not using it.