We have an issue when trying to show 24 live/playback streams at the same time on a single HTML page. Every camera image (canvas) that shows frame images is scaled so 24 streams can fit into a single HTML page. The customer has requested that all streams are shown in full HD so we can provide them with digital zoom functionality on every camera that is shown on the screen. The main problem, in our opinion, can be that we receive full HD images and redraw those images on significantly smaller canvas sizes. That can be task with a high CPU requirements and hard to execute on the web browser. To be able to fulfill our customer requirements we have the following questions.
1. When the HTML page size is changed, that involves a change of the canvas that shows the frames. How we can in live or playback, while receiving frames from the stream, change the resolution of the frame that we need?
2. How to handle digital zoom functionality over the mobile server without requesting full HD images for every camera? How to request part of the full HD images with top/left offsets and defined with and height, while we are already connected to the stream?
3. While receiving frames from the mobile server, how we can receive information is camera connected/disconnected/have any connectivity issue? The main problem is that when the camera is disconnected we still receive the same frame from the mobile server, without any other information that the camera has a connectivity issue. Is it possible to receive that information while we are connected to the stream over the mobile server?
Below is a short part of the code that we use for implementing the solution. This pattern is taken from one of the Milestone examples that came with milestone installation. We would appreciate it if you can provide us with instructions on how to accomplish those requirements. If you have any advice on how we should connect and handle streams, we are keen to follow your advice and modify implementation so we can provide the customer with the requested functionality.
The code sample is below:
---------------------
var cameraId = id; // This is parameter that we receive from the Milestone
var signalType = 'Live'; // Or 'Playback'
var inWidth = this.width; // Width of canvas dom element
var inHeight = this.height; // Height of canvas dom element
var streamRequestObj = <any>{
CameraId: cameraId,
DestWidth: inWidth,
DestHeight: inHeight,
SignalType: signalType /*'Live' or 'Playback'*/,
MethodType: 'Push' /*'Pull'*/,
Fps: 25,
ComprLevel: 71,
KeyFramesOnly: 'No' /*'Yes'*/,
RequestSize: 'Yes',
StreamType: 'Transcoded', //'FragmentedMP4', // 'Transcoded'
// SrcWidth: inWidth,
// SrcHeight: inHeight,
};
if (this.isPlaybackMode) {
toRet.PlaybackControllerId = this.playbackController.id;
}
var videoConnectionReceivedFrame = (frame) => {
if (frame.dataSize > 0) {
if (frame.hasSizeInformation) {
var multiplier = (frame.sizeInfo.destinationSize.resampling * XPMobileSDK.getResamplingFactor()) || 1;
this.image.width = multiplier * frame.sizeInfo.destinationSize.width;
this.image.height = multiplier * frame.sizeInfo.destinationSize.height;
}
if (this.imageURL) {
window.URL.revokeObjectURL(this.imageURL);
}
this.imageURL = window.URL.createObjectURL(frame.blob);
this.image.src = this.imageURL;
} else {
//empty frame
}
}
this.videoConnectionObserver = {
videoConnectionReceivedFrame: videoConnectionReceivedFrame
}
var requestStreamCallback = (videoConnection) => {
this.videoController = videoConnection;
videoConnection.addObserver(this.videoConnectionObserver);
videoConnection.open();
this.camera_relatedAudioGuid = this.camera_relatedAudioGuid,
this.videoConnection = videoConnection;
}
this.streamRequest = XPMobileSDK.RequestStream(streamRequestObj, requestStreamCallback, function (error) {
console.warn('Error on request live stream.');
});
// Option for playback
// this.streamRequest = XPMobileSDK.RequestStream(streamRequestObj, requestStreamCallback, function (error) {
// console.warn('Error on request playback stream.');
// });
this.image = document.createElement('img');
this.image.addEventListener('load', onImageLoad);
this.image.addEventListener('error', onImageError);
var onImageLoad = (event) => {
var zoomedImage = scaleIt(this.image, this.canvasDomElementContext.canvas);
this.canvasDomElementContext.drawImage(zoomedImage, 0, 0, this.canvasDomElementContext.canvas.width, this.canvasDomElementContext.canvas.height);
}
var onImageError = (event) => {
console.warn('Error on image loading.');
}