The following is my understanding of how camera and clip timestamps work. There are some non-intuitive aspects, and it varies based on camera, codec, and camera configuration.
A video clip commonly has three separate timestamps: 1. Creation date at the filesystem level, 2. Creation date at the internal metadata level, 3. Encoded date at the container level. #2 and #3 can be viewed with MediaInfo or Invisor (available on the Mac App Store).
The filesystem creation date is normally stored internally as UTC in the camera card filesystem, and after offloading to a computer, it is displayed adjusted for the local time zone at the computer's location. If you shoot the clip at 13:00 EDT then view it on a computer in the US central timezone, it will show as 12:00 CDT.
This creation date at the internal metadata level is normally the same as the filesystem creation date, but is not adjusted for local timezone. In tools like MediaInfo it is expressed as date/time in the time zone where the clip was shot.
The encoded date is expressed as UTC in tools like MediaInfo, and it can be the same as the metadata creation date, minus UTC adjustment.
In some other cases, depending on camera configuration, the encoded date/time derives from the timecode generator (or external timecode), which could then be different from the creation date/time if the camera time-of-day clock differs from the timecode clock. In such cases the camera timecode clock is normally set to "free run" timecode.
This is not limited to high-end cameras. Even a mirrorless camera like the Sony A7SIII has separate timecode clocks and time-of-day clocks. It can be set to free-run timecode. In such cases the metadata encode date can be very different (even a fractional hour different) from the camera time-of-day clock.
Were all the clips shot on the same camera? If they were shot on different cameras, the settings on those cameras could be different. If shot on the same camera in the same mode, normally I'd expect the timestamps to be consecutive, not jump around. Depending on the camera type, if it was powered down for a significant period when changing the battery, the time of day or timecode clocks could be different.
The approximate difference between 16:00 and 20:00 is +4 hr, or the difference between UTC and EDT. It's almost conceivable the 16:xx timestamp was EDT, and the 20:xx time was encoded date expressed as UTC.
The best way to start investigating is pick a clip of interest and export the data using MediaInfo or Invisor, post that here then we can examine it further. If would help if we knew the camera make and model, what the recording codec and file extension is, and the MediaInfo report. To obtain this, install MediaInfo, right-click on the video file, open in MediaInfo, examine with View>Text. Then export the report via File>Export and pick "Text" as the export format.
