Milestone VPS - removing a stream/camera doesn’t free the GPU RAM memory

We are using the Milestone VPS plugin with Deepstream Pipeline that includes different Detection/Classification Models. When we connect a camera through Milestone everything is working fine except when disconnecting it. Upon disconnection, the reserved GPU memory is not freed which leads to the accumulation of reserved but unused memory. This is what is printed on the IVA/Pipeline side:

“ReceiveLoop websocket exception: The remote party closed the WebSocket connection without completing the close handshake.”

In a big system with many connections/disconnections happening, this leads to full GPU memory and the crash of the system.

Can you please assist on this?

Please note that we are using Milestone 2020 R2 and VPS From MIPSDK 2020 R1. Milestone is running on a Windows OS while the VPS/Deepstream Pipeline is running on Ubuntu 18.04.

The log is from the server running VPS/Deepstream pipeline.

Also note that this issue is happening whether we manually delete hardware from Milestone or there was a disconnection between the Milestone and the VPS/Deepstream server (e.g. Milestone server down).

Hi Fares,

Could this be an issue with GC, are you able to get a memory dump after disconnect to determine what is holding onto the GPU memory?

kind regards

Simon

There is a problem in the sample code when disconnecting. Can you try to replace your two files “vps2gstreamer.h” and “vps2gstreamer.cpp” with the ones that I have attached.