TransmitMessage is taking time from component integration and client plugin for multiple messages over federation

MIP
, , , ,
1

I am developing a solution that will utilize the TransmitMessage to send bulk data in multiple chunks from MIP Component to Smart Client Plugin.

My solution has support of the federation, where child site component sample sends bulk data in multiple chunks to the Parent site Smart Client plugin.

I am facing the delay of 1-2 minutes to receiving the whole chunks of the data at client side.

Could you please provide suggestion to improve the latency.

Sample Code Chunk:

Component Sample (Runs on the Child Site - Server)

MessageCommunicationManager.Start(EnvironmentManager.Instance.MasterSite.ServerId);
messageCommunication = MessageCommunicationManager.Get(EnvironmentManager.Instance.MasterSite.ServerId);
 
private object MyMessageHandlerAsync(VideoOS.Platform.Messaging.Message message, FQID destination, FQID sender)
{
 
 m_logger.WriteDebug($"Message received, client object id: {message?.ExternalMessageSourceEndPoint?.ObjectId}");
 
	var resultList = GetResults();
	var totalRecords = resultList.Count;
 
	if (totalRecords > 0)
	{
		int batchSize = 1000;
		for (int i = 0; i < totalRecords; i += batchSize)
		{
			var dataObject = resultList.Skip(i).Take(batchSize).ToList();
			
			var result = JsonConvert.SerializeObject(dataObject);
 
			m_logger.WriteDebug($"[PARTH] Send, Data Count: {dataObject.Count}, size : {result.Length}");
 
			messageCommunication.TransmitMessage(new VideoOS.Platform.Messaging.Message(receiverMessageId, result), message.ExternalMessageSourceEndPoint, null, null);
						
			dataObject.Clear();
		}
	}
	resultList.Clear();
}

Logs

2024-12-11 16:13:22.9404 Message received, client object id: 9741ab4f-8d34-442d-bce1-94f18a2bb35d

2024-12-11 16:13:23.0535 [PARTH] Send, Data Count: 1000, size : 343987

2024-12-11 16:13:23.1978 [PARTH] Send, Data Count: 1000, size : 344226

2024-12-11 16:13:23.3301 [PARTH] Send, Data Count: 1000, size : 344214

2024-12-11 16:13:23.4706 [PARTH] Send, Data Count: 1000, size : 344169

2024-12-11 16:13:23.5874 [PARTH] Send, Data Count: 1000, size : 344311

2024-12-11 16:13:23.7191 [PARTH] Send, Data Count: 1000, size : 344373

2024-12-11 16:13:23.8378 [PARTH] Send, Data Count: 1000, size : 344518

2024-12-11 16:13:23.9391 [PARTH] Send, Data Count: 1000, size : 344288

2024-12-11 16:13:24.0378 [PARTH] Send, Data Count: 1000, size : 344261

2024-12-11 16:13:24.1598 [PARTH] Send, Data Count: 1000, size : 344018

2024-12-11 16:13:24.2962 [PARTH] Send, Data Count: 1000, size : 344313

2024-12-11 16:13:24.4303 [PARTH] Send, Data Count: 1000, size : 344175

2024-12-11 16:13:24.4303 [PARTH] Send, Data Count: 163, size : 56229

-------

Client MIP Plugin Code (Runs on the Smart Client Plugin - Parent Site)

public object MyMessageResponseHandler(Message message, FQID destination, FQID sender)
{
	private List<ResultData> responseData;
	
	if (message != null && message.Data != null)
	{
		responseData = JsonConvert.DeserializeObject<List<ResultData>>(message.Data.ToString());
	}
 
	 m_logger.WriteDebug($"[PARTH] Received, Data Count: {responseData.Count}");
	
 
	return null;
}

Logs

2024-12-11 16:13:38.9599 [PARTH] Received, Data Count: 1000

2024-12-11 16:13:38.9685 [PARTH] Received, Data Count: 1000

2024-12-11 16:13:38.9825 [PARTH] Received, Data Count: 1000

2024-12-11 16:14:25.5511 [PARTH] Received, Data Count: 1000

2024-12-11 16:14:25.5611 [PARTH] Received, Data Count: 1000

2024-12-11 16:14:25.5993 [PARTH] Received, Data Count: 1000

2024-12-11 16:14:25.6233 [PARTH] Received, Data Count: 1000

2024-12-11 16:14:25.6561 [PARTH] Received, Data Count: 1000

2024-12-11 16:14:25.7062 [PARTH] Received, Data Count: 1000

2024-12-11 16:14:25.7573 [PARTH] Received, Data Count: 1000

2024-12-11 16:14:25.7844 [PARTH] Received, Data Count: 1000

2024-12-11 16:14:25.8405 [PARTH] Received, Data Count: 1000

2024-12-11 16:14:25.8628 [PARTH] Received, Data Count: 1000

2024-12-11 16:14:25.9055 [PARTH] Received, Data Count: 163

Summary of issue:

As you can see from the above logs there is significant delay in receiving the message form client to server

  1. Server has sent
    1. first message chunk timestamp 16:13:23.0535
    2. last message chunk timestamp 16:13:24.4303
  2. Client has received
    1. first, second, third message chunk timestamp 16:13:38.9599 (Delay of 15 seconds)
    2. Fourth-last message chunk timestamp 16:14:25.5511 (Delay of 58 Seconds from Sever to Client)

Note:

  1. Apart from this other code chunk in my solution is also utilizing TransmitMessage for exchange of custom data.
  2. On the same site this delay is negligible :slight_smile:
2

First of, the message communication is not meant as a file transfer mechanism so you might need to reconsider whether this is the right channel for your needs. However, this does not explain the huge delays you are seeing, and tbh I cannot see any other reason for it than either network issues or heavy load on the Event Server used for the communication. Maybe try have a look at performance counters as well as the log files of the Event Server to see if anything shows up there.

3

Hello Peter,

  1. Could you please suggest the right channel to transfer the data from component integration on event server to smart client plugin?
  2. As you can see from the logs above at receiver side
    1. First three chunks received at 16:13:38 second timestamp
    2. Rest 11 chunks received at 16:14:25 second timestamp
    3. Looks like it stuck somewhere.
    4. Could you guide how to trace this?
  3. I have checked the performance counters and logs but not found anything.
    1. image
      image895×480 51.9 KB
4

XProtect does not provide a file transfer mechanism as such and within reasonable limits the Message Communication channel should also be able to handle it.

You can set TraceMessageCommunication on EnvironmentManager (https://doc.developer.milestonesys.com/html/index.html?base=miphelp/class_video_o_s_1_1_platform_1_1_environment_manager.html&tree=tree_search.html?search=trace) to true from your plugin and component integrations to get some more log information - hopefully that will show something.