performance considerations regarding multiple playback videos in browser

MIP
, , ,
1

We want to display multiple playback recordings in (any) browser.

we went over the API documentation and the demo apps.

we found two versions:

1. MIPSDK\ProtocolSamples\MobilePlaybackSample\DemoApp2

2. The demo that is installed on the server, :8081/DemoApp/index.html. (depreciated)

questions:

1. Is there a way that we can directly access the (playback) video stream, rather than frame-by-frame images?

2. The deprecated DemoApp uses web sockets, while the newer version (DemoApp2) does not. Can we use web sockets with the API demonstrated in DemoApp2 ? If so, can you give us an example or explain how to do it?

3.Given that we need to display multiple recordings in the browser at the same time, do you have any recommendations regarding performance best practices?

2

Hi Roy,

First of all I have to apologize cause we haven’t make it clear that all the DemoApps are deprecated.

We noticed that is not very clear in the documentation, but there is available MIP SDK Mobile API .

It has very handy and tiny JS implementation.

All the new samples are provided trough it.

It continues to evolve and provides most easiest way for Web integration.

It can be downloaded from here:

https://www.milestonesys.com/my-milestone/msp/mip-sdk/

(Look for MIP SDK Mobile)

Samples (JS/Web) are included in the latest Mobile server (12.3 - 2018 R3).

The one relevant to you is available at “<mobile_server_address>/XPMobileSDK/Samples/Video/videoSample.html”.

(By default it is disabled, please search the forum how to enable it).

Now to the questions:

  • “Is there a way that we can directly access the (playback) video stream, rather than frame-by-frame images?”

I’m not completely sure what exactly you have in mind.

Do you mean to access raw DB written frames ? For example if they are H.264.

Or want to play a little bit smoother?

  • Web sockets are supported in all the modern browsers as well in the Mobile server from huge amount of releases.

MIP SDK Mobile (for Web) uses them out-of-the-box, without any need from your side to configure anything. You can disable them (in the SDK) purposely, but usually Integrators do not do it.

  • “Given that we need to display multiple recordings in the browser at the same time, do you have any recommendations regarding performance best practices”

Actually when displaying multiple video streams in a browser usually the bottleneck is in the Mobile server itself, cause it will doing a trans-coding of every stream. We haven’t noticed significant performance limitations in browser, even displaying 9 steams simultaneously for example.

Can I ask what exactly will be UX when you display multiple recording at the same time ?

And in particular do you need all those recordings to be synchronized between each other or not ?

If you need something similar (we call it Multi-Camera Playback) we could give you directions how to achieve it (basically let the Mobile server to do the synchronization and simplify the client side code as much as possible, eg just displaying/presenting particular camera images).

3

Thank you Petar for this detailed and excellent response.

I am now experimenting with the new mobile API. I believe I read somewhere that the mobile API can connect to a single server. Can this single server access payback recordings (not live) that were recorded by other servers that share the same data storage?

Indeed, I was thinking of simply accessing a H.264 video stream, for example, and setting the src of a

Our uix is the following:

The page will display numerous recording tiles related to a past incident: Videos from milestone cameras recorded by multiple milestone servers, along with screen recordings and phone call recordings from other sources. All of these will need to be able to play:

individually or synchronized.

faster/slower

in reverse.

I would be happy to learn more about multi camera playback with server-side synchronization.

4

Hi Roy,

I’ll try answer your questions.

  • Can this single server access payback recordings (not live) that were recorded by other servers that share the same data storage?

Mobile Server (MoS) can access all the recordings that are available from the SC. In other words if there is an installation, where there is a single Management Server (MS) and multiple Recording Servers (RS), MoS that is connected to the MS can access recordings for all the RSs. Same is valid and for the multi-site installations. If the MoS is connected to the “master” it will be able to access all the recordings from the “slaves”. It is more matter of configuration not the actual data persistence. Or physical recordings placement.

  • Indeed, I was thinking of simply accessing a H.264 video stream, for example, and setting the src of a

Unfortunately you can not access the raw recording directly from the Mobile server. It is not intended for that. I’m not sure how you will be able to play them directly in the video tag. As far as I know they should be multiplexed to something that browsers understand - for example MP4 file chunks (could be used something like “mp4box.js”). If you plan to use MSE of course. Otherwise you should make them HLS “compatible”.

  • I would be happy to learn more about multi camera playback with server-side synchronization.

In order to be achieved synchronized multi-camera playback, first have to be created “playback controller” (XPMobileSDK

.createPlaybackController()). The way is very similar to what should be made in .NET MIP SDK (if you are familiar with it) . After that on every creation of the stream (XPMobileSDK.requestStream()) is given the id of already created playback controller. The controller is used further to “control” the playback (speed, direction) ,as well as to be get information for the current playback time. The individual stream from other hand are used only to present video frames.