We have a client that has several hundreds of cameras deployed with their XProtect Corporate install. They now need to deploy an AI Bridge so IVAs can better integrate and leverage XProtect.
While I understand that it’s recommended that AI Bridge run on NVIDIA-certified hardware running the EGX platform, what I don’t know is how many feeds AI Bridge can handle per machine, especially given that hardware can be sized with differing amounts of CPUs, RAM, GPUs, etc.
What I want to avoid is “trial and error”, i.e. making an educated guess, deploying the hardware, then finding out that it’s insufficient, resulting in additional expenditure and time lost procuring additional hardware. Obviously I’d like to run AI Bridge on a single machine (i.e. a single-node K8s cluster), but if that isn’t possible, then I’d like to know ahead of time.
Can anyone help with this? I’ve been unable to find any information on this matter.
Thank you in advance for any help.
It is very difficult to give an accurate answer on this. It all depends on the what kind of cpu, gpu, networking, memory, video streams, analytics and so on, that will be used. In many cases the main bottleneck will be the GPU which is used by the analytics applications (not AI Bridge).
The IVA being developed to use AI Bridge is being sent to NVIDIA’s Validation Labs for some profiling to determine performance, scalability, etc. So I’m less concerned about that.
The feeds will be coming from 1080p cameras (likely H264 and/or H265), obviously brokered through XProtect.
This seems like something of a “chicken and egg” issue, since we’re asking what AI Bridge requires to run at scale given basic parameters like number and type of feeds, and we’re being told it depends on the CPU, GPU, memory, etc. of the system AI Bridge is to run one, which of course will depend on what AI Bridge needs in order to run given the parameters we gave, etc.
With respect, there must be some guidelines that can be given in terms of hardware specifications to run AI Bridge on, even if it requires gathering the info Milestone needs to help make that determination. AI Bridge is a Milestone product, and so I’d expect some guidance or guidelines in terms of how to size the necessary hardware at scale, other than just “NVIDIA-certified EGX systems”. It’s a black box to non-Milestone people, so I’d hope that we could rely on Milestone to help determine hardware sizing (apart from “trial and error”), even if it’s in the form of some kind of basic scaling formula or the like.
Put another way: For those customers that have already installed AI Bridge, how did they determine the appropriate hardware sizing they needed? Did they just do “trial and error”? Or was there a more precise method used?
Thank you as always for your help and (continued) patience.
Actually, this entire question on how to dimension a system to work with a given number of cameras is something we have been fighting with for many years in terms of XProtect itself. We have created many different generations of server estimator tools each of them having their own set of weaknesses. There are so many parameters in play and we have had a couple of bad experiences in the past sharing load expectations based on number of cameras. In general we don’t want to promote specific partners but they all provide systems with different performance characteristics and therefore it is really difficult. One way we have solved this for XProtect is to offer a Husky Appliance box where we know all the performance details. Here we can tell you exactly what you can expect. Maybe, it could also be a good idea to have a similar AI ready box that we know all the performance characteristics of. This could become a sort of reference system. Otherwise, we in general rely on our integrators to know the details about what systems are good for what and how to scale to them to meet certain demands. So the integrators will build up this knowledge from project to project and they will recommend whatever they have good experience with. When it comes to AI, it is still a new field for many of our integrators and it will be a bit of trial and error in the beginning. Milestone is however not involved in this process unless of course something is not working as expected. I do recognize the challenge, and I will bring forward the need for being able to come with more precise recommendations.