My company currently outputs RTSP streams from milestone VMS into a 3rd party platform that leverages AI models and returns metadata and possibly stream output as well. Were planning a way to ingest the returned video stream and or metadata back into VMS. (were talking bounding boxes, metadata for alerts, so forth) From what I understand using the AI-bridge is the best solution for this course.
I know that if we plan on ingesting a stream profile, we will need a license for each returned stream from our IVA. But if it’s just metadata, we should not need another license - just wanting to confirm on that.
I’m wondering if this is true too - If we decide to just use metadata, if structured appropriately out of the AI bridge, the xprotect smart client with draw bounding boxes itself (based off of the metadata). But if we decide to stream, what ever annotations that get returned from out of the IVA, the video stream will be preserved in the recording server.
Is my thinking correct on this? Are there any other major ideas I’m missing here?
Hi and thanks for your question and let me try giving an answer!
The “best Architecture” is of course always relative to the scenario you want to support and the tools and platforms you want to create the solution with..!
The AI Bridge is built for supporting scenarios like the one you describe – especially if your analytics application is running in Docker containers. The AI Bridge bridges between existing open XProtect VMS APIs and API formats that are typically used in the Intelligent Video Analytics (IVA) community, like RTSP, ONVIF, REST, Kafka etc.
Thanks for your response and it does help clear up some questions. And I think going the way of the AI bridge does make the most sense. Our 3rd party IVA is in docker containers and have infrastructure in place already that ingest stream straight from VMS into IVA using open bridge.
So I would just need to use the AI bridge for ingress of metadata. Since this 3rd party IVA does not have a structure in place to communicate correctly with the AI bridge, my plan is to develop a middleware/adapter service that will essentially become the official IVA app to communicate with the AI bridge.
3rd party video analytics via HTTP > HTTP receiver + Middleware/Adapter (IVA APP) > AI bridge > xprotect VMS.
Do you see any potential issues with this infrastructure?
This sounds good, and an adapter layer should do the trick. Do you plan to continue using the Open Network Bridge for RTSP streaming, or will you also move that to AI Bridge?
Make sure you get the right timestamp from the videoframes, and use that timestamp again on Metadata frames and Analytics Events and potentially videoframes ingested back into XProtect. And yes, you will need a license (only) for each returned Video stream from your IVA.
That’s a good question. It stands to reason that it would simplify the infrastructure and just use one type of bridge. Are there any other benefits that you can see by merging all our output streams in the AI bridge as well?
AI Bridge is designed to run and serve consumers inside a secured network like Kubernetes. If you need more widely and user authorized RTSP streams, Open Network Bridge is the right bridge to use.