Hello,
We overlay camera analytics (AXIS Scene Metadata) on live video retrieved with the MIP SDK, and we need to match each metadata frame with the right video frame. We measured a constant offset of about 260 ms between the two, and we would like to understand how the Recording Server timestamps live video, and whether we can get the camera capture time instead.
Setup
-
XProtect Express+ 2025 R3 (build 17737), MIP SDK 25.3.2 (.NET Framework 4.8, x64), single server (management + recording)
-
Extension modules : Axis On Screen Controls Plugin, Axis Optimizer v6.0.10.0
-
Camera: AXIS P3747-PLVE, Axis OS 12.11.77, added with Axis driver (XProtect VMS Device Pack 13.8a)
-
Stream : H.264 2592x1944, 8 fps, keyframe every second (driver default), Zipstream low; metadata device enabled (Scene Metadata + events in the metadata stream)
-
Video: RawLiveSource, GenericByteData 0x0010 packets (32-byte header, one frame per packet)
-
Metadata: MetadataLiveSource; we use the UtcTime of tt:Frame in the XML
-
BeginTime seems to be UtcTime made monotonic, so we do not use it, correct ?
-
All machines and the camera are NTP-synchronised to the recording server (< 2 ms, checked with w32tm)
What we measured
- Overlay offset: the boxes fit the images only if a metadata frame with UtcTime t is matched with the video frame whose Milestone timestamp is about t + 250 to 300 ms.
- To find the cause, we read the camera directly over RTSP, in parallel with the MIP SDK client, for 60 s, with the ONVIF replay header extension (0xABAC) requested for that session only. The camera serves the same encoded stream to XProtect and to us, so 453 of 478 frames were matched byte for byte. For each frame, with t = capture time from the extension:
- frame received directly from the camera: t + 247 ms
- same frame received by our MIP SDK client: t + 240 ms
- timestamp in the GenericByteData header: t + 264 ms (263 to 265 ms over 60 s)
- Scene Metadata UtcTime = t: in the direct stream, boxes and images match with no shift
- The camera is therefore consistent. The offset is the camera’s capture-to-emission latency (~250 ms), plus the fact that the live timestamp is set close to the arrival time at the Recording Server, not at capture. The frame spacing follows the camera’s RTP timestamps (regular 50 ms grid), as explained in this thread: How are timestamps assigned to each frame of video captured from cameras in XProtect?
- The live timestamp is slightly later than our own reception time (by 7 to 90 ms depending on the session), and this difference changed after a Recording Server restart (-29 ms before, -7 ms after). We also have the impression that the overlay of boxes in the Smart Client improves when the server has been running for a long time.
- Enabling Network.RTP.OnvifReplayExt=yes on the camera (for all streams) and restarting the Recording Server did not change anything. As far as we can see, the documentation only mentions the 0xABAC extension for Edge Storage Retrieval with the ONVIF driver.
Questions
- How exactly does the Recording Server compute the absolute timestamp of live frames from an Axis camera? Arrival time plus RTP spacing? Is the anchor set when the device connects, smoothed over time, or corrected periodically? Is it expected to converge after the server has been running for a while?
- Is there a setting (Management Client, driver, Recording Server configuration) to timestamp live video with the device time: RTCP sender reports, ONVIF replay extension, or RTSP Range clock?
- Would the ONVIF driver behave differently for live video?
- Is there a way, through the MIP SDK, to get the device capture time or the RTP timestamp of each live frame (header field, other source class, or other API)?
- What is the recommended way to align camera analytics metadata (camera UtcTime) with video frames? Does the Smart Client apply any compensation when it draws bounding boxes?