We are using h264 cameras and wish to show them in a webpage (html5). We have tried the mobile server but we are experiencing that the FPS is not steady at 25fps. We would like to try h264 raw live video from the recording server to the browser. Does anyone know how to do this?
I have left this unanswered hoping that other partners might have valuable input.
There is no method or sample in the MIP SDK that fits..
I framed it a bit different with the hope that your Bulgarian web&mobil team would answer the question with a different answer than you provided Bo. My conversation with them on the Milestone Developer Conference suggested that this had already been tested by their development team.
Hi Havard,
If you can use RTSP stream, you could use the Onvif Bridge Server. There is no need to implement the whole Onvif protocol logic. If you can get the camera IDs with the Mobile Web SDK, the he could request the video stream from the Onvif Bridge server via link rtsp://:@:554/live/
I hope this helps!
Hi Harvard,
We really had tested similar solution before.
It was based on the proprietary “HSL like” protocol.
It was however difficult for implementation both from server and client sides.
And we consider it as not really successful.
Now we are making research on another technology, based partly on the WebRTC.
It is however in the earlier stage, although it is very promising.
So for the moment I’m not sure we could propose you robust solution for that.
Btw one of our colleagues (Tsveti) proposed following:
“The partner could use the Onvif Bridge Server. There is no need to implement the whole Onvif protocol logic. If he can get the camera IDs with the Mobile Web SDK, the he could request the video stream from the Onvif Bridge server via link rtsp://:@:554/live/”
Although it sounds crazy on first look it could have value in it.
You know WebRTC is based on sRTP, so if you implement custom (reverse) proxy which to convert RTP to sRTP and implement (WebRTC) communication wrapper,
it could work.
It seems like a huge amount of work however.
I have looked at webRTC and it looks promising.
Have G-streamer 1.14 (includes webRTC) been tested?
Unfortunately no.
Hello everyone. I just wanted to know if Milestone has anything planned to support integrators that would like to stream the video directly to modern web browsers? In the past we would just wrap your SDK into ActiveX, but this option is dead and today we neeed something more than SDK or RTSP stream from ONVIF Bridge. We have recently successfully created a robust web player for live and archive video playback from Avigilon ACC as they recently introduced a plugin called Web Endpoint Service. It’s basically a web service API that allows (among other things) to query the system for cameras, recordings metadata and initiate stream playback (live or archive). Because the stream is in fMP4 format with H264 encoding, we can control it from javascript level and decoding is being handled by the browser. For all other major VMS vendors we need to have a proxy service that will handle the RTSP format and translate it, but we really like situation where we don’t need it and we have one possible point of failure less. So far only Avigilon made it possible to skip the proxy but we were thinking who else will follow? 
If you need any information about this use case please let me know I will provide all details and possibilities how you can improve your solution. Thank you!
Hello, just wanted to push up this thread in hope someone will anwser it. I’m willing to elaborate more on the topic if there is anyone here to discuss this with.
Hi Jakub,
Could you contact me on e-mail with regard to that request ?
(Or provide your in registration I’ll write to you 
Hi Jakub,
which proxy service do you use?
Have you made it yourself?
@Petar Vutov I just sent you an email
@Havard Ruden Helland At first we made a simple proxy ourselves but it was not very stable and since we are not video experts we didn’t want to proceed this way. Then we started testing third party proxies. We found a few that fit into our architecture, proof-of-concept worked, but then we made this integration with Avigilon that doesn’t require one! This is a way we would like to go, as a proxy adds an unnecessary point of failure, delay and extra cost. I also think that using a proxy is a temporary solution as vendors will provide better support for web playback and those components will not be needed at some point. This is the reason why I’m contacting Milestone as their system is very closely related to our PSIM (GEMOS) and we don’t want its integration to be left out.
@Petar Vutov and @Jakub Bartkowiak We find ourselves in exactly the same situation as you did 2 years ago. Jakub, you predicted the vendors would be addressing this soon but it’s been 2 years and we dob’t see much progress in this field. Did you end up finding a Milestone solution or did you invest further in your own proxy?
Gavin, we developed this middleware which was mentioned “proxy” here to solve the problem with HTML5 video streaming from Milestone VMS. Live and archive video.
Have you tried the direct streaming?
This solved our issues.
Hi @Gavin Hill, back then we have received some experimental version of Milestone SDK which contained samples for retrieving and displaying video directly in modern browser (I believe it was using websocket for transport) but apart from it being experimental, we still couldn’t use it in our enterprise app. The SDK was using global variables and was not modular, it would not allow us to stream more than one video or from more than one server (I don’t remember now exactly) and conflict our modular/scoped architecture. We have provided Milestone developers with concrete hints about architectural changes needed so this SDK can go beyond just a sample, but we never heard back from anyone about this.
As we needed to solve this issue and couldn’t use this experimental SDK, we asked our friends at Viinex to create a proxy which would allow us to stream live and archive video from XProtect and show it in our browser based application. @Andrey Kireev can tell you more about this solution.
@Håvard R Helland could you write more about your use case? First of all I see this method only allows to stream live? which leads me to believe the stream only comes from camera and not from recorded storage? Second of all, is this documented how can this be used in third party app? Thank you.
As Havard mentioned, since then Milestone officially supports so called Direct Streaming. In short you can fetch live video in form of fragmented MP4 files. Supported camera encodings are H.264 and H.265. In the browser those MP4 fragments could be presented via MSE (Media Source Extension), but only if the codec is H.264. The only exception in the browsers world is the Mac’s Safari - it could display H.265 also.
DS has some limitation however:
- Works only in live mode (e.g. cannot be used for recordings)
- Works only on higher product types (Corporate and Expert)
- Has a delay of around 2 second (between when frame is created in the camera and is displayed in the browser). Sometimes a little bit lower (1 to 2 s)
It is used by the XProtect Web Client as well as by XProtect Mobile clients. But it is available for usage from integrations also.
There are mainly two possible ways of integration:
- Via MIP SDK Mobile
As Jakub mentioned there are some limitations in how the Web SDK is created (not formatted as packages, single instance, etc).
- Via protocol integration towards the Mobile server
Although the nice wrappers won’t available in this way, the Mobile server protocol is simple and most of it could be reverse-engineered. There are few tricky moments like DH Key Exchange, Binary format of the video channel, etc, but in general is not a rocket science. Of course there also documentation available (for the protocol).
I’m not aware of any limitations regarding how many streams could be pulled simultaneously from the Mobile server. The MIP SDK Mobile for Web fetches video via WebSockets, in order to overcome the limitation of 6 simultaneous HTTP requests per end-point. The mobile server itself could serve over 200 direct streams on I7 (7th gen) based desktop machine.
If you want to go in the direction of your custom “web” server, I would advice you to look at the WebRTC. And in particular, as Havard mentions, on the GStreamer’s implementation. It is in useable state since 1.16.2, but is more stable in the 1.18 (version of the GStreamer).
WebRTC is presented more easily in the browser, and has significant advantage of lowering the latency to the astonishing 250 ms. Of course the complexity of such solution is significant.
Thank you @Petar Vutov for the update on current situation, it’s good to know what is available. Unfortunately, we still won’t be able to integrate our product with XProtect directly, because Direct Streaming has two major limitations (from our perspective): lack of low latency live streaming and archive streaming. We don’t have these problems currently because Viinex proxy allows us to access XProtect archive and streams over WebRTC, which as you pointed out, is the best solution available at the moment for low latency live streaming.
That is very interesting @Jakub Bartkowiak !
Could you share some details about the Viinex proxy solution ?
For example what implementation of WebRTC they are using, how they are fetching the video from the Milestone VMS, how they implement the “playback” functionality, as there is no built-in into the WebRTC protocol (I would suppose they have would have something like “playback controller” in their proxy service). In all the cases it sounds very exciting !