XProtect Expert – Scalable Way to Fetch 1 FPS Snapshots from 600 Cameras (6 Recording Servers)?

General
1

Hello Milestone Community,
We are working with a client using Milestone XProtect Expert with:
6 Recording Servers
Around 600 cameras
Our requirement is to retrieve 1 snapshot per second from each camera (approximately 600 snapshots per second total) for an external AI surveillance system.
Currently, we are able to:
Fetch snapshots successfully using REST API and SOAP API
Run our AI processing pipeline on the retrieved images
This works well in testing with a limited number of cameras.
However, our main concern is scalability when running this continuously across:
6 Recording Servers
600 cameras simultaneously
We are very concerned about system stability, load distribution, and long-term performance impact when operating at this scale.
We would like guidance on the following:
Is polling the REST/SOAP API at 1 FPS per camera a recommended approach for production?
What is the best way to manage snapshot requests across multiple Recording Servers?
Should we distribute requests directly to each Recording Server instead of routing everything through the API Gateway?
Is there a more scalable architecture recommended by Milestone?
Are there known performance limits or best practices for this type of high-frequency image retrieval?
We want to ensure we design the integration properly and avoid overloading the Recording Servers or other components.
Any official recommendations, documentation references, or real-world experience would be greatly appreciated.

Thank you in advance!

2

The API Gateway’s Cameras task = JpegGetLive uses the same underlying mechanism you asked about earlier in this topic. (RecorderCommandService JpegGetAt ao.)

After discussing this with a developer, here are the key points:

  1. High load on the Recording Server (RS):

    Generating even a single snapshot consumes significant RS resources. If every RS produces one snapshot per second, the number of cameras a single RS can support will be greatly reduced.

  2. API Gateway behavior:

    The API Gateway itself performs very little processing; it simply forwards the request to the RS. However, this can create high bandwidth usage if everything is routed through a single API Gateway.

    It is possible to install and use multiple API Gateways to distribute the bandwidth usage.

  3. Minimizing RS load:
    The lowest impact on the RS is achieved by requesting the raw live video stream and decoding it on the client side.

    • With the ImageServer protocol, you can obtain the raw live video.
    • By default, cameras using H.264 (or similar) produce one keyframe per second. This means you can extract that keyframe to obtain one still image per second—unless the camera’s settings are changed by the user.
    • Alternatively: Using the MIP SDK (MIP .NET Framework library), the decoding is done client‑side, which offloads work from the RS.

  4. Performance testing:

    Milestone has not conducted testing on performance or load when making many frequent Cameras task=JpegGetLive requests through the API Gateway.