In addition to Ian's great advice:
It appears that you are changing the position and scale of items over time
This creates keyframes. You can edit these keyframes using the Inspector, the Viewer, and the "Video Animation".
One particular characteristic of such keyframes - specifically for position - is that they can be either "Linear" or "Smooth".
What you are describing suggests that some keyframes are set to "Smooth", and you'd probably need them to be "Linear". This can be changed directly in the Viewer, but it perhaps easier in the Video Animation.
I had a clip to which I added two keyframes directlty in the Viewer.
To adjust the keyframes, select the clip in the timeline and press Control-V. Keyframes appear as little diamonds:

You can see there "Transform: All". Click the little down arrow, and select Position:

Now if you control- or right-click on each diamond, you can select "Linear":

Also, if you accidentally created extraneous keyframes, you can delete them by selecting "Delete Keyframe".