Well the evidence so far is that most cameras don't have a particularly accurate time-stamp, but the real issue for us is dealing with the variability. When extracting from the OSD we discovered that most seconds have 8-9 frames (the camera is set to 10FPS), and then you get occasional craziness with a second containing 20 frames or less than 5.
If the camera wrote the time it thought a frame was captured into the stream, it would remove a lot of guesswork, and reduce the computation load (computer vision is expensive) of dealing with the video.
Ideally we would like to rely solely on using video captured elsewhere, without installing new kit or software. But based on this we might not have a choice.