video playback skips a few minutes when user doesn't interact with paused player in Safari

Whenever I pause the video and go to another tab, when I go back to the video and click play, it will skip several minutes ahead, if not more than a dozen depending on how long I leave it “paused". This happens only in Safari (using the latest build released with MacOS Ventura) when I playback videos on Crunchyroll, Udemy or Platzi video players.

This issue comes from a few safari versions ago (I can reproduce this issue since Safari 14)

Steps to reproduce:

1.  Play any video for a minute and pause
2. Open a new tab, wait 5-20 minutes 
3. Go back to the tab with video
4. Click play

Expected result: Video starts to play from the timestamp where it was paused

Actual result: Video starts to play from wrong timestamp (where you pause + time you was absent).

Noting that if you use Chrome or Firefox user agent within Safari, video players of mentioned websites works as expected.