What causes the Server Command Service Get Configuration command to take 10+ minutes to respond?

MIP
, , ,
1

We’re having an issue retrieving the server configuration from Milestone during peak hours and at the busiest time it takes 10+ minutes to download the configuration from the management server.

We use this API to download the configuration:

/ManagementServer/ServerCommandService.svc

Using the SOAPAction:

http://videoos.net/2/XProtectCSServerCommand/IServerCommandService/GetConfiguration

We have a simple script that runs every hour and downloads this configuration to detect changes that we need to be aware of (such as cameras added or switching to a failover recording server).

The log file for our script shows the time it finished running, so we can compare how long it took each time. Throughout the night, as the server is less busy, the script that downloads the file runs faster, running in a minute at 3 am.

Why does it take so long to retrieve this configuration? Is there a way to get this configuration more efficiently?

2

How big is your installation? Can you please explain it to us?

3

It’s a Milestone Interconnect Management Server with 8000+ cameras across 30+ recording servers. There are around 3 million past alarms stored on the machine. We pull the entire XML file from Get Configuration a few times each day and the file is around 1 MB.

4

I suspect a general performance issue.

Please test on the same PC with the same user, when the user logs in on the Smart Client, is the Smart Client ready faster?

You cannot expect the load of configuration to be faster on your own client that on the Smart Client, if the Smart Client has the same issue please ask the Support Community for advise.

https://supportcommunity.milestonesys.com

5

We checked last night and the Smart Client loaded instantly while get_configuration() through the protocol integration took 10+ minutes. I’ve attached the machine’s CPU and RAM. What else could be causing this?

image 2
image 2367×480 82 KB
image 3
image 3543×480 66.8 KB
image
image313×480 73.4 KB
Here’s the python code that is communicating with the management server:

def get_configuration(server_ip, ssl_port, basic_username, basic_password):
    
    v = uuid.uuid4()
    
    data= '<?xml version="1.0" encoding="utf-8"?><soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"><soap:Body><GetConfiguration xmlns="http://videoos.net/2/XProtectCSServerCommand"><instanceId>' + str(v) + '</instanceId><currentToken></currentToken></GetConfiguration></soap:Body></soap:Envelope>'
    data_len = len(data)
    data_len = str(data_len)
    
    base_user_pass = base64.b64encode('[BASIC]\\{}:{}'.format(basic_username, basic_password))
    
    headers = {
        'Host' : str(server_ip),
        'Content-Type': 'text/xml; charset=utf-8', 
        "Authorization": "Basic " + base_user_pass,
        "Content-Length": data_len,
        "SOAPAction": "http://videoos.net/2/XProtectCSServerCommand/IServerCommandService/GetConfiguration",
        "Connection": "close"
        }
    try:
        response = requests.post('https://{}:{}/ManagementServer/ServerCommandService.svc'.format(server_ip, ssl_port), headers = headers, data = data, verify= False)
        return response.content
    except Exception as e: 
        logging.error("unable to retrieve server information with exception: {}".format(e))
        get_configuration(server_ip, ssl_port, basic_username, basic_password)
        return
6

As this works fine at 3 am, it must be a performance issue. In general the Support Community will have a much better feel for performance issues but will have no idea if the Smart Client works fine every time.

Some ideas; as quick questions:

Do you have a federated setup with multiple sites available?

Do you have slave servers (e-code)?

7

The integration is through Interconnect (c-code), so yes it has a federated setup with multiple sites. We connect to the Interconnect machine that has the Management Server for logging in and downloading the get configuration xml. There is no noticeable latency when we connect to any of the offsite recording servers.