I have a gstreamer/deepstream pipeline code written in python and running on a Linux server. The pipeline does object tracking and other inferencing on the detected objects. I want this pipeline to integrate with VMS such that it reads from and reports to the VMS system.
According to this link https://doc.developer.milestonesys.com/html/gettingstarted/intro_vps_toolkit.html, I can use VPS plugins such as vpsboundingboxes to process video frames from an XProtect camera and return results back. If I want to customize the vpsboundingbox plugin to contain my gstreamer/deepstream pipeline logic (or develop a new plugin), I should add cpp code or is it possible to reuse my python code somehow?
In case VPS plugins can only be developed in cpp, can I use my python pipeline to treat the XProtect stream as rstp, execute my pipeline, and then stream back results via rtsp protocol also using the universal driver capability? In this case, can I still send back bounding boxes and other metada from my pipeline? What could be the limitations?
Currently we do not support python, so I’m afraid you would need to convert your plugin to cpp, unless there’s a way to call that python pipeline through cpp through gstreamer.
As for the second question, there are many ways and one way could be to use onvif bridge to get rtsp stream, process it and then either use VPS driver or make your own driver using MIP driver framework to inject metadata back to XProtect. Inside XProtect you can link metadata from one device to another one to be able to display bounding boxes.
Thanks @Artur Michael Gadomski (Milestone Systems) , I’ve chosen to go with converting my plugin to cpp. In this custom plugin, I am using:
fromxprotectconverter
queue:src’ connected to the ‘src_metadata’ pad of the fromxprotectconverter
fakesink:connected to the above queue
queue:src’ of the queue connected to the ‘src_video’ pad of the fromxprotectconverter
h264parse:connected to the latter queue
nvv4l2decoder
nvstreammux
Other nvidia plugins..
When the XProtect camera is connects to my plugin, I get the following and everything hangs afterwards:
0:07:32.068834397 76 0x7f4cc0003ed0 WARN h264parse gsth264parse.c:1237:gst_h264_parse_handle_frame: broken/invalid nal Type: 5 Slice IDR, Size: 7 will be dropped
0:07:32.068847829 76 0x7f4cc0003ed0 WARN h264parse gsth264parse.c:1237:gst_h264_parse_handle_frame: broken/invalid nal Type: 2 Slice DPA, Size: 27 will be dropped
0:07:32.068857950 76 0x7f4cc0003ed0 WARN h264parse gsth264parse.c:1237:gst_h264_parse_handle_frame: broken/invalid nal Type: 29 Invalid, Size: 179 will be dropped
0:07:32.192218867 76 0x7f4cc0003ed0 WARN h264parse gsth264parse.c:1237:gst_h264_parse_handle_frame: broken/invalid nal Type: 2 Slice DPA, Size: 17794 will be dropped
It seems that h264parse gstreamer plugin is not able to properly parse the stream. I tried the sample plugins provided in the VpsSamples and they are working fine. For example, the vpsboundingboxes returns bouding boxes that appear in the camera.
The bounding boxes one doesn’t actually do any decoding, just generates bounding boxes. Does jpeg transcoder plugin work? Can you generate the pipeline out of that plugin to see what plugins decodebin is using?