Video from custom gstreamer pipeline is not shown in Mangement Console / Smart Client

MIP
, , ,
1

I adapt vpsjpegtranscoder plugin to use my own plugin

  1. fromxprotectconverter
  2. decodebin
  3. videoconvert
  4. xprotectjoin ( with meta sink from 1. )
  5. myplugin
  6. jpegenc
  7. toxprotectconverter

When I change *myplugin* to *identity* I have the video inside Smart Client.

When I use *myplugin* I see that _chain function executes normally, but there is no video inside Smart Client.

Even worse, when I make the _chain function of myplugin to just forward buffer to srcpad I don’t have any video inside Smart Client as well, but in this case it should be exactly the same behaviour as identity plugin or GST_VPSPASSTRU.

2

Hi Anton.

I see you have already found a solution to your problem of no video in Management Client in this question:

https://developer.milestonesys.com/s/question/0D53X00007aE9Q8SAK/video-from-custom-gstreamer-pipeline-is-not-shown-in-mangement-console-smart-client

Do you still have an issue or has it been solved?

Best Regards,

Kristine

3

Good day Kristine,

I still have some issues and questions about VPS.

We are trying to get the stream from VMS using VPS integration method.

When I send metadata to VMS they are discarded, because the return stream is 5 seconds old.

We tested vpspasstru, vpsboundingboxes, and vpsjpegencoder.

The differences between VMS stream h264 from IP camera and the stream returned from VPS are:

vpspasstru < 1 second

vpsboundingboxes around 1 second

vpsjpegencoder around 5 seconds and is stable

We investigated the pipeline of vpsjpegencoder with GST SHARK.

It turns out the total proctime is below 1 second for all elements of the pipeline, but latency is around 5 seconds.

We tried to add queue leaky=2 before decodebin and jpegenc that has changed nothing.

We found out that delays are connected to IP camera fps

With 25 fps delay is below 1-2 seconds

With 10 fps delay is around 5 seconds

Questions:

  1. Do you have any ideas about how it is connected?
  2. If I take vpsjpegtranscoder pipeline, a plugin that doesn’t touch video (just pushes gstbuffer further), but adds some metadata (like vpsboundingboxes) and I put the plugin before header_inserter I can see video, but no metadata. If I remove header_inserter and finish my pipeline with plugin I will get metadata, but no video (which is fine without header_inserter). The question is why with header_inserter the metadata of plugin just before it doesn’t go to VMS?
  3. What is a good way to send nothing to VMS from the pipeline? Can I just make a fakesrc as a ghostsrc of the pipeline?

We can provide the images of the pipeline and GST SHARD profiles on demand.

The time difference between VMS and VPS is less than 1s

Hardware:

VMS Professional + 2020 R2

(Windows 10, RAM 8GB, CPU: i5-4310)

VPS

(docker container on Linux RAM: 8 GB, Intel(R) Xeon(R) Silver 4214R CPU @ 2.40GHz)

Machines are in the same local network

4

1. I’m having trouble getting vpsjpegtranscoder plugin to run in our nvidia test environment, I will keep trying to reproduce the scenario over the holiday break. But could it be that decoding takes a while to get started? How long is the GOP on your video? Have you tried the nvidia plugin that does decoding using some different plugins and is there also a delay? What about just setting a ‘now’ timestamp on the frames?

2. header_inserter takes the information about frames like timestamps and sequence numbers and creates a generic byte data header to wrap frame data in. It’s required to get video back to XProtect in a format it understands. When I was trying out and put vpsboundingboxes plugin after header_inserter, I could both see the transcoded video and bounding box on top of it. When I put the vpsboundingboxes plugin before header_inserter I had a lot of problems. I think it’s because both plugins strip whatever gstreamer metadata is there in a buffer and adds their own without copying the rest as it should. It’s not a problem with vpsboundingboxes as the last plugin because by that time header_inserter has already read the frame information and converted the video. I will see if I can make vpsboundingboxes preserve the existing gstreamer metadata information.

3. That works fine. I tried it out with vpsjpegtranscoder where I put fakesink after header_inserter and fakesrc as ghostpad.