How to set up microphones on Mac mini for Apple Intelligence?

I like Apple Intelligence. Makes some stuff really easier, but it's annoying that I can't use a regular microphone on a Mac Mini seamlessly with it.

"Listen for Siri" only shows up on a Mac Mini in Settings after connecting AirPods.

While Apple Voice Control is working with a regular USB microphone, Apple Intelligence is not, which makes no sense to me.

To use Apple Intelligence with a regular microphone you have to use a keyboard shortcut and then click on the little microphone icon every time you like to ask Siri something, which is just annoying.

[Re-Titled by Moderator]

When using Apple Voice Control you can just speak your commands without pressing any extra buttons.

I've even ordered AirPods 4 and Apple Intelligence is working flawlessly with them, but only when I wear them all the time and that's not what I want.

Funny part is that other microphones are sometimes working with Apple Intelligence (after connecting AirPods) for one or two tries but that's it.

Obviously I know how to add and set up microphones in settings.

Is there an update coming that will fix this?

Or any other help?