How stream video live and recorder) in web browser from milestone server?

I ask for your help to integrate milestone products in my web application. because of I’m new in the video and milestone world.

I want to show a video(live/recorded) from milestone server in my web application.

Usually when I show a video in my webapplication i have an url and I encode in an HTML media player.

I’m looking the way I can get the stream from milestone server, this is what I found till now:

1)SDK Mobile

I tried but what if the customer doesn’t want install the mobile server for security reason?

The SDK Mobile do the login, ask a stream and put on an HTML canvas the frame image. It works but is limited and I don’t have great control of it.

2)ONVIF Bridge. I don’t it very weel but is it suitable for my porpose? I can have access to the recorded video? Is it compatible with all the camera in the milestone server? Is it easy to install in an enterprise architecture milestone server?

3)Should I create an asyncronus video stream API through the SDK Component that transcode to some html compatile stream? If yes, do you have any example?

4)Can I get the stream throw the new milestone REST API? If yes how?

5)What about WebRTC?

Do you know other way to show a video from milestone server to my WEB application?

What do you suggest to explore?

Someone has experience about similar integration?

I would like to avoid the SDK Mobile because is limited and the customer could be not available to install the mobile server.

Thank you very much in advance

Hi Alessandro,

Let me try to address all the questions, although it is possible to miss something.

Mobile server and MIP SDK Mobile for JS/Web seem to me your first and natural choice.

Mobile server is designed exactly for that purpose - to provide easy VMS access for Mobile and Web clients. XProtect Web Client uses it.

I 'm not sure what would be the “security” reasons that would force customer to skip installation of the Mobile server. Actually it is the other way around.

Mobile Server is initially designed to work in unfriendly environment of the Internet. It has build-in protections for most of well-known web attacks (like XSS, SQL injection, etc). As well as is built using best practices in the field - fully sanitizing all the inputs and outputs for example.

I believe Mobile Server is a must for your integration.

It provides access to Live and Recorded video streams, where they are transcoded to series of JPEGs. In addition for the Live video is available so called Direct Streaming, which is basically CMAF / MP4 streaming, played in the browser trough MSE.

Transcoding is universal and works for all the camera codecs. The drawback is higher CPU/GPU load of the Mobile server as well as possibly higher bandwidth on higher resolutions and frame-rates.

Direct Streaming just repacks the original video frames from the cameras and could work with JPEG and H.264 on all the browsers and with H.265 on Safari and (eventually) Edge*.

In addition there are plenty of samples how to integrate video from Mobile server with the MIP SDK Mobile. For both Transcoding and Direct Streaming.

Milestone Open Network Bridge (formerly known as ONVIF Bridge) consist two services - ONVIF one and RTSP one. I would suggest that you are more interested in the RTSP part, as video could be fetched only trough it. (ONVIF protocol/interface is mostly for configuration)

RTSP service supports all the cameras of the systems with H.264 and H.265 codecs. Probably soon and with MJPEG.

And you can fetch both Live and Recorded video streams.

But as you know you cannot integrate RTSP/RTP steam directly into the Browsers.

So this is some option if you want to fetch the video from the VMS and convert it to other Web friendly format later (those are not much btw).

You can install Open Network Bridge as easy as the Mobile server. Mean time of installation (of both) is less than a minute without affect of the “core” VMS components. No any system down-time, etc.

I’m not sure I fully understand your question here. I would suggest you are talking to some form of HLS or MPEG-Dash (in terms of HTML 5 compatible streaming).

If you want to do something like that, you could try few slightly different approaches:

  • Fetch video from the Mobile Server with Direct Streaming (for live only). You will receive directly MP4 files which should be compatible with both HLS and MPEG-Dash. You will need to take care only for Playlists and synchronization. (Why you wouldn’t like to display them directly with the Browser’s MSE is completely different question)
  • Fetch RTSP stream from Open Network Bridge, or fetch video directly from RS (IS protocol integration). And put that to some tool that is doing HLS/Dash repackaging.

Please bear in mind, that in all the cases you will have some delay playing those HLS/Dash streams. If you do conversion wisely, you will be able to achieve around 5 seconds delay (in best case 3 seconds). But in most cases is much more - like 15 to 30 seconds. Which is extremely important (and could be unacceptable) for Live streaming.

As far as I’m aware of Milestone REST API is exposed for Configuration only. And you cannot fetch any media trough it.

Unfortunately Milestone doesn’t provide built-in WebRTC support.

You could try to build such support by yourself.

There are few tools / libraries that could be used for that purpose.

From the unpaid ones the natural choice would be GStreamer.

You could fetch the video from the VMS (possibly trough RTSP) and push it to the WebRtcBin. And you should take care about signalization.

But if you are not very familiar with GStreamer, as well as with WebRTC (from how it works perspective), probably is not a good idea to go in that direction.

Ext)

As I mentioned there are not much options to present video in the browser nowadays:

  • Transcode to series of JPEGs (Mobile SDK/Server)
  • Media Source Extension (Mobile SDK/Server)
  • HLS/MPEG-Dash (Your own implementation)
  • WebRTC (Your own implementation)
  • RTMP or other Plug-in (Your own implementation) - although Flash / Plug-ins seem already dead to me (RIP)

At least I’m aware only of those options.

It get’s a little bit longer, but hopefully helpful.

Hello,

I would appreciate an updated answer for that question with respect to new capabilities in latest 2022R3 version and the future 2023R1.

Thanks,

Hi Alexander,

I don’t believe something has changed in 2022 R3 (as integration possibilities).

As for the upcoming 2023 R1 - I could comment after the official GA, which would be next week (I believe).

Hi Alexander,

2023 R1 is already out, and as you can see in the API GW documentation WebRTC has initial official support.

And it seems my previous answer were not exactly correct. Because in 2022 R3 there is beta support for WebRTC in the API GW.

So you could give it a try (WebRTC I mean)