how to disable pay with password and use only face id

Stolen Device Protection does require Face ID/Touch ID without the use of the Passcode to make any purchases or access cards in your Wallet. Turn it on and you are protected.

• Face ID or Touch ID biometric authentication: Some actions such as accessing stored passwords and credit cards require a biometric authentication with Face ID or Touch ID — with no passcode alternative or fallback — so that only you can access these features.

About Stolen Device Protection for iPhone - Apple Support

Defending against in-person physical duress is a challenging proposition.

Adding another authentication barrier would just make the criminals force any additional credentials out of the victim.

The most effective defense in that sort of threat environment is to simply not have the exploitable info on the mobile device.

What DOES inhibit Wallet Transactions is remote activation “Lost Mode.”

You can send feedback to Apple here:

Product Feedback - Apple