Apple Watch Series 4 Screen keeps enlarging

I have only had the Apple watch series 4 for a week and intermittently, the screen enlarges and you cannot see the face of the clock. The only way to clear it is to turn off the watch. Does anyone know if this is a hardware issue or am I doing something wrong?