Jitter behaviour when consuming stream using the github code example of protocol integration

MIP
, ,
1

Hi

we experience Jitter behaviour when consuming stream using the github code example https://github.com/milestonesys/mipsdk-samples-protocol/tree/main/TcpVideoViewer

Basically, we run that Milestone’s code example of consuming Milestone stream and the stream is very jerky compared to the ‘preview from the Milestone XProtect Client program’.

we use Milestone XProtect Management Client 2020 R2 (Version 20.2a Build 4122)

here is the attached screenshot where

  1. Red color is official code example from github
  2. Blue color is preview from the Milestone XProtect Client
  3. Green color is VLC

https://xrvisionpteltd-my.sharepoint.com/:i:/g/personal/natanaela_cynapse_ai/EZvacePe8iZOkwRzJajjAf0BnBEMCrHru5A_tLysk3ALZA?e=5rXeTm

and the video snapshot of the issue

https://xrvisionpteltd-my.sharepoint.com/:v:/g/personal/natanaela_cynapse_ai/EUmrYS5TrrZGgUSgNx4FicgBCP3R9z1KD-MkQ2sV5pQuYQ?nav=eyJyZWZlcnJhbEluZm8iOnsicmVmZXJyYWxBcHAiOiJPbmVEcml2ZUZvckJ1c2luZXNzIiwicmVmZXJyYWxBcHBQbGF0Zm9ybSI6IldlYiIsInJlZmVycmFsTW9kZSI6InZpZXciLCJyZWZlcnJhbFZpZXciOiJNeUZpbGVzTGlua0NvcHkifX0&e=ND5ryT

2

The TcpVideoViewer asks for transcoded images in JPEG format no matter what codec the camera in the XProtect VMS is setup to utilize. This does not give optimal performance.

It is possible for you to implement a solution where you ask for the original raw images, and you develop an optimized decoder and rendering engine.

To get raw instead of transcoded data please use in the Image Server protocol, no and no - https://doc.developer.milestonesys.com/html/index.html?base=reference/protocols/imageserver_request_response.html&tree=tree_search.html?search=alwaysstdjpeg

3

Thank you very much for your answer.

1. If we ask for the original raw image, what is the original raw format of the image? Is it YUV or other format?

2. Do you have the code example which converts this original raw format into for example Bitmap or any other format?

4

The format is encapsulated data, within it the data is governed by the original camera setting which can potentially be a lot of things as many video codecs are supported.

To understand the data please see “The GenericByteData format” - https://doc.developer.milestonesys.com/html/index.html?base=reference/protocols/genericbytedata.html&tree=tree_3.html

No, unfortunately there is no sample like that.

5

We are a computer vision company who needs to stream many cameras using Milestone VMS and our system is docker container based in linux system.

We just want to make basic connection to Milestone Server and we are trying the code example from the TcpVideoViewer code example.

We already tried to set tag to “no” and remove the “transcode” tag, but the issue still persist.

We have past project which used (NET Framework) Milestone SDK implementation, and the camera stream seems to be better compared to the protocol integration in linux. Is the protocol integration the best way to consume stream from Milestone Server in linux system or is there any other options?

6

When you make the change can you verify that you get h264 or whatever the camera is setup to use instead of JPEG? The sample will not work as it cannot display anything but JPEG, but it will prove that you can get the raw and unaltered video.

If you can use Net Framework you will have an easy access to a lot of functionality that is not as easy when dealing with the protocols.

Behind the scenes, the MIP library and other ways, all use the Image Server protocol and in that sense it is the best.

The Mobile Server is created and optimized for delivering and transcoding to JPEG in the size that the client requests but also include a direct streaming offer. The Mobile SDK has samples that are not .Net..

https://doc.developer.milestonesys.com/mipsdkmobile/

If you had a new server, not 2020 version, you also had an option of using WebRTC..

https://doc.developer.milestonesys.com/html/index.html?base=samples/protocolsamples/webrtc_javascript/readme.html&tree=tree_3.html

7

From Milestone XProtect 2022 R1 you can use the Milestone AI Bridge to bridge to Intelligent Video Analytics (IVA) applications deployed as docker containers.

Milestone AI Bridge forwards video streams (RTSP or gRPC) from cameras added to the XProtect VMS to the IVA applications for video analysis. Milestone AI Bridge allows the IVA applications to send the analysis results back into your XProtect VMS as analytics data (events, metadata, and video).

See more here Architecture overview and system components - Third-party integrations | Milestone Documentation 2023 R3 (milestonesys.com)