How to switch to another payment method for an already running subscription?

I had one payment method added originally.

I want to discontinue that payment method and switch to a new one.

I was able to add the new payment method successfully.

Now, I want to remove that old payment method. I tried to delete it, it said that there are active subscriptions using this payment method, so it cannot be deleted.

Then I tried to see if I can switch all existing subscriptions to the new payment method, but on the subscription screen there is no option to view the payment method or change it.

So I tried to cancel the subscriptions, in case that makes them "inactive" and allows for my payment method to get deleted. But that did not help either. My subscriptions are cancelled, but they are still active for some days / months depending on the subscription.

How is one supposed to fix this issue?

Note: I'm a developer and into tech, so please feel free to go into technical details.

[Edited by Moderator]