XProtect integration architecture — POC vs. Component/Plugin vs. AI Bridge, looking for guidance

MIP
1

Hi,

We’re building a customizable AI object & action detection integration for XProtect (EyePop.ai) and are trying to decide on the right architecture before we go through Milestone verification. Wanted to get your read on this before we commit engineering time in one direction.

Where we are today

We have a working mvp built on the MIP SDK Driver Framework. It streams each enabled camera a second time to a local “bridge” process, which forwards it to our cloud inference service and returns ONVIF metadata for bounding boxes. It works, but it has some real drawbacks we’re not comfortable shipping as-is: each analyzed camera ends up as two separate hardware entries in XProtect (one for recording, one for us), the camera is streamed twice, there’s no installer or Windows service (the bridge is started manually).

We also scoped a different architecture: a MIP Component integration (a Windows service connector using the MIP .NET SDK, reading live video via RawLiveSource) plus a full Plugin integration across Management Client, Event Server, and Smart Client. That gets us a single MSI installer, one hardware entry per system (not per camera), native rules/alarms support, and configuration that lives entirely in XProtect’s own config. It’s a significantly bigger build.

While comparing these, we came across Milestone AI Bridge, which looks like it might replace the need for either approach: it reuses the camera’s existing stream (no duplicate hardware entries), pushes analytics data back via a standard broker (REST/gRPC/Kafka), and gives us native rules/alarms and Management Client configuration without writing a custom driver or plugin. We have read on the developer forums that there is a lot of latency with this setup.

What we’re wrestling with

  1. Is AI Bridge considered a supported, verifiable path today, on par with a classic MIP driver or Component/Plugin integration for the “Milestone Verified” program?

  2. Given we’re building what’s essentially an IVA, is there a reason a distributor or Milestone’s verification team would prefer one of these three paths over another, beyond what’s in the public docs?

  3. If we do go the AI Bridge route, is there anything about the verification process itself that changes, timeline, requirements, testing, compared to the driver/plugin path?

Happy to share our current architecture write-ups if useful.

Thanks,
Andy
EyePop.ai

3

I can provide a little insight for you on this… I’ve done something very close if not the same to what your planning to do, using the AI Bridge. We use a 3rd party platform that provides AI analytics/metadata and have been able to ingest metadata into our VMS, which then allows it to propagate to the recording server, which then allows real time bounding boxes to be recorded.. etc.. etc..

Overall, for us, a high overview infrastructure looked like this:

  1. Export video via Open Network Bridge to 3rd party AI inference

2. Build a IVA to ingest metadata from 3rd Party AI, that then gets Processed to send to AI bridge

3. AI Bridge ingest metadata and sends to VMS.

There are 3 ways you can ingest data into the AI bridge as you will see in the documentation: Events, Metadata, Video. Sounds like you want Metadata. Video will cost you as importing a video stream is considered another device.

Pros: The AI bridge itself isn’t slow. Depending on your metadata load I guess, I can tell you I’ve had over 400 cameras ingesting metadata into the bridge and when it worked, it worked. I ran with the previous version 3.0.0. The only latency will come from your inference AI platform, if your IVA is efficient. I had roughly a 100ms-250ms lag if I remember correctly. It wasn’t instant, but did its job. You can run Docker or Kubernetes depending on your needs. I had much more success with Docker.

Cons: If you have a lot of cameras that you want to subscribe metadata to, the processing plugin side for Management client can get a bit squirrely. As it creates a metadata device for every camera subscription that you have to manually subscribe to.

Now what I’m going to say next, is something we experienced and could very well be our own issues we had with our infrastructure and possibly some issues may have been fixed with the newest version of 3.0.1. The milestone developers could not replicate our issue we had. But in our case, it wasn’t stable at such a high number of subscriptions. Adding a metadata device would sometimes cause an intermittent issue with our recording server that would bring it down.

In our case we eventually just shelved it, as it was causing too much downtime for us in accordance to this problem. Its very hard to create a testing environment of hundreds of cameras to proceed safety. We might visit this again one day, as we are slowly making our recording server more efficient and ironing out some of the issues we had with it.

To reiterate, this was just my experience with it, I believe due to our own infra. I think its a great product and will probably only get better as more versions are released.

I’m sure a Milestone Dev can properly respond to this post and answer more of your technical questions you may have.

Hope this helps:

1 Like
4
  1. Is AI Bridge considered a supported, verifiable path today, on par with a classic MIP driver or Component/Plugin integration for the “Milestone Verified” program?
  • Yes, AI Bridge is considered a supported!
  1. Given we’re building what’s essentially an IVA, is there a reason a distributor or Milestone’s verification team would prefer one of these three paths over another, beyond what’s in the public docs?
  • No, AI Bridge is targeted IVA apps running on Linux Docker or Kubernetes
  1. If we do go the AI Bridge route, is there anything about the verification process itself that changes, timeline, requirements, testing, compared to the driver/plugin path?
  • No
5

Thank you very much for sharing this.

6

Milestone AI Bridge bridges the well-known Protocol Integration APIs to

  • RTSP (live) and gRPC (live and playback) streaming APIs

    • This means that generally you won’t need the Open Network Bridge in IVA setups

  • GraphQL API for stream discovery and App registration

  • REST APIs for Video, Metadata and Analytics Events injection

You can access the AI Bridge via the Milestone’s Amazon Public ECR and the Milestone Download Page.

7

Thanks @HansK! Although that part of the egress infrastructure was set up before I came in, I believe it was the way they wanted it to work with that 3rd part AI inference company, I just took care of the ingress video metadata ingestion coming back, which means the “IVA” or really it was just a middleware - received in MQTT messages, formatted it for milestone, and sent it off to the bridge, organized per customer - each with a docker instance. We might visit the new version soon in testing.