We have installed ONVIF Bridges on XProtect Professional+ 2024 R2 and are using the RTSP service.
We obtain video from the RTSP client as follows:
For live streaming:
rtsp://:@:554/live/
For playback:
rtsp://:@:554/vod/ Range: clock=Z-
At this time, there are cases where the first RTP packet sent from XProtect does not have a PTS (Presentation Time Stamp) set.
In particular, for playback, the first RTP packet never has a PTS set. For live streaming, there are also cases where the PTS is not set on the first RTP packet, though this does not happen every time.
I would like to know if this is by design.
Furthermore, if the first RTP packet does not have a PTS set, the RTSP client cannot determine the correct playback start timing because it does not know the “presentation time” of the first frame.
As a result, the beginning of the video may be skipped or playback may be delayed.
Therefore, please let us know how to ensure that the PTS is set from the first RTP packet.
Best Regards.
Additionally,
When receiving RTSP streams from XProtect using ffmpeg, the following logs are output, but video reception is successful.
ffmpeg -rtsp_transport tcp -i rtsp://:554/vod/ -loglevel debug -c copy output.mp4
[rtsp @ 0x5578dacc1140] first_dts 43200 not matching first dts NOPTS (pts NOPTS, duration 6000) in the queue
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[mp4 @ 0x5578dadb74c0] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
[mp4 @ 0x5578dadb74c0] Non-monotonous DTS in output stream 0:0; previous: 0, current: 0; changing to 1. This may result in incorrect timestamps in the output file.
As this shows, there is a possibility that the timestamps may be inaccurate if the first packet of the RTSP stream does not have PTS/DTS set.
Please let me know how to configure the RTSP server (XProtect) to assign correct PTS/DTS values.
Hi @Kouzou Matsuyama
Described behavior looks like a bug, but is it something that blocks your work?
Streaming should allways start, despite the warnings in log.
I don’t think you can do much to alter the current product behaviour.
Thank you for your reply. I would like to ask a bit more.
If PTS/DTS are not set, it becomes problematic because the video cannot be played back or analyzed in the correct order and timing.
If possible, I would appreciate it if you could let me know the conditions under which this occurs and any possible workarounds.
Best Regards.
Sorry, I don’t have a good enough answer for you at this point.
A workaround might be to use time from elsewhere for your analysis, or wait for the second package.
I’ve filed a bug which will hopefully be fixed in the upcoming releases.
Regards.
Thank you for your response.
Could you please explain the reason why you determined it to be a bug (for example, a violation of the specification) and your policy for fixing it?
I would like to confirm that the fix in XProtect will not affect the decoding side that receives the video.
Best Regards.
Hi @Kouzou Matsuyama
You reported it as a problem. It sounded yo me you want this fixed in the product, therefore I filed a bug for further investigation of the underlying cause. I don’t know whether it’s a violation of the specification (I doubt it) but it sounds logical that any RTP package will contain proper timestamp data.
I cannot give you any timeframe for the fix, at this point. All fixes are tested internally before being released, therefore you shall not expect any problems with it.
But if you found a workaround and don’t need any fix for this and prefer the current behavior, that’s fine by me. Just let me know.
Best regards.
Thank you for your response.
I would like the product to be modified so that appropriate PTS(Presentation Time Stamp)/DTS(Decoding Time Stamp) values are set for every H.264 frame.
Hi,
PTS and DTS are related to mp4 not RTP. The RTP header contains a timestamp that can act as PTS but no DTS info is provided by specification. I ran a wireshark trace on a playback stream from Open Network Bridge and I see each frame having timestamp filled in the RTP header.
Could you please provide more info what is wrong on the RTP stream?
KR,
Svetlana
We are developing a function that analyzes video streams using AI by utilizing the RTSP feature of XProtect.
In the RTSP stream from XProtect, the first RTP frame does not have PTS information set, which can cause decoding to fail with an “Invalid pts” error on the receiving side.
For example, when receiving an RTSP stream from XProtect with ffmpeg version 5.16, the following logs are output:
cur_dts is invalid st:0 (0) [init:1 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
[mp4 @ 0x562adff011c0] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
[mp4 @ 0x562adff011c0] Non-monotonous DTS in output stream 0:0; previous: 0, current: 0; changing to 1. This may result in incorrect timestamps in the output file.
As shown above, the packets in stream 0 do not have timestamps (PTS/DTS) set.
This is deprecated and may stop working in the future, so it is necessary for the sender to set the timestamps correctly.
Best Regards.
Hi ,
When referring to missing PTS, can you please explain to which field in the RTP header (https://datatracker.ietf.org/doc/html/rfc3550) you refer. A wireshark trace representing the problem can be very useful.
Is it sth that happens all the time or just sometimes?
Thank you
When referring to “missing PTS (Presentation Time Stamp is missing)”, it usually means that the “Presentation Time Stamp (PTS)” assigned to video or audio frames does not exist.
This PTS is not directly the same as the “timestamp” field in the RTP header (RFC3550).
I understand that, in principle, the RTSP server should assign correct timestamps (DTS/PTS) to each media packet.
For reference, I have attached the debug log result obtained by connecting to the XProtect RTSP server (192.168.1.150) from 192.168.1.151 using ffmpeg, as well as the WireShark capture obtained at the same time.
ffmpeg -rtsp_transport tcp -i rtsp://:@192.168.1.150:554/live/80a6db53-5d1e-4458-bb16-5cab6371b41e/live -loglevel debug -c copy output.mp4
The following output in the log is relevant.
[rtsp @ 0x562145cd1200] first_dts 43200 not matching first dts NOPTS (pts NOPTS, duration 6000) in the queue
Best Regards.
The timestamps in rtp are set as follow:
- Relative timestamp in RTP packet
- Absolute time for synchronization with relatioin n to the relative one in RTCP packet.
In the wireshark you send those are all provided:
So, I do not see missing timestamps.
For further investigations - I can suggest , you
- try to play the respective stream with VLC, ODM , so you can isolate weather it is tool related.
- try with another camera , one that is at least 10 fps, so you can identify weather it is specific camera related issue
It is not the timestamp in the RTP packet, but the PTS (Presentation Time Stamp) and DTS (Decoding Time Stamp).
PTS is the time information for the order in which the video is displayed.
DTS is the time information for the order in which the video is processed.
The first packet of the stream sent from XProtect does not have PTS/DTS assigned.
Since the RTSP client performs playback and processing based on the received PTS/DTS information, it is essential that the RTSP server(XProtect) sends correct information.
Best Regards.
Hi,
In the wireshark provided by you, we can see that all rtp packets have a timestamp (relative), also RTCP packets with synchronization info are send regularly.
In order be able to help you further , could you please open a support case?
Kind regards,
Svetlana
