Apple Watch not reading heart rate continually during workout

I'm using apple watch ultra in the last few months now. My heart rate reading have a lot of gaps during workout. Both in running and in swimming.

I previously add apple watch 7 and the reading was much more stable.

I read some feedback that it is related to the outside temperature but I'm not sure it is the case.

I can't find any correlation between the outside temp and the reading, it worked fine in 0 deg C and didn't properly work at much higher temp. The gaps are not only at the beginning but all along the workout, picture attached.

Is it an hardware problem in my device?