Test code gets error "Could not start direct streaming" with MobileSDK web.

I am using Milestone Server 2020R3, direct streaming is enabled on the Mobile server and the “VideoStream” sample code works fine.

The test code that I wrote does the following after the window load event:

1 - automatically connects to the server.

2 - once connected, automatically logs in.

3 - once logged in, goes through the views and creates a videos-stream node and adds it.

When doing it this way, I get a “Could not start direct streaming” message. I put in some extra addeventhandlers to capture events from the videos-stream to see what was happening. In there, it logged a lot of “beginVideoStuck” events, and eventually CloseStream was issued.

When I have it connect, login, display video through a button click, it works. The video creates a lot of “endVideoStuck” events though. in the “VideoStream” example code, it does not have any “beginVideoStuck” or “endVideoStuck” events.

Below is my single page test code.

<!DOCTYPE html>
<html>
 
<head>
  <link rel="stylesheet" type="text/css" href="lib/css/sample.css">
  <script type="module" src="lib/modules/VideoConnection/main.js"></script>
  <script type="module" src="lib/modules/VideoStream/main.js"></script>
  <script src="lib/js/polyfill-webcomponents-bundle.js"></script>
  <script src="lib/js/XPMobileSDK.js"></script>
</head>
 
<body>
  <section id="login-form-container">
    <div>
      <label>
        <span class="label-text">Server</span>
        <input type="text" class="server-name input" name="server" id="server" value="https://localhost:8082" />
      </label>
      <label>
        <span class="label-text">Username</span>
        <input type="text" class="username input" name="username" id="username" value="testacct" />
      </label>
      <label>
        <span class="label-text">Password</span>
        <input type="password" class="password input" name="password" id="password" value="password" />
      </label>
      <div>
        <button type="button" class="login-button button" id='connect'>Connect</button>
      </div>
 
    </div>
  </section>
</body>
<section id='streams-container'>
</section>
<script>
  var observer;
  var credentials;
 
  document.querySelector('#connect').addEventListener('click', StartLive);
 
 
    window.addEventListener('load', function()
    {
      StartLive();
    });
 
  function StartLive() {
    var tserver = document.querySelector('#server').value;
    var tusername = document.querySelector('#username').value;
    var tpassword = document.querySelector('#password').value;
    var creds = {
      url: tserver,
      username: tusername,
      password: tpassword
    };
    initMilestone(creds);
  }
 
  function initMilestone(cred) {
    credentials = cred;
      if (observer) {
        XPMobileSDK.removeObserver(observer);
      }
 
      observer = {
        connectionDidConnect: connectedToServerHandler,
        connectionDidLogIn: loggedInHandler,
        connectionLostConnection: lostConnectionHandler
      };
      XPMobileSDKSettings.MobileServerURL = credentials.url;
      XPMobileSDK.addObserver(observer);
      XPMobileSDK.connect(credentials.url);
  }
 
  function connectedToServerHandler() {
    XPMobileSDK.login(credentials.username, credentials.password, {
      SupportsAudioIn: 'Yes',
      SupportsAudioOut: 'Yes'
    });
  }
 
  function loggedInHandler() {
    LoadVideo();
  }
 
  function lostConnectionHandler() {
    console.log("Connection to server lost");
  }
 
  function streamError(event) {
    console.log("Stream Error: " + event.type);
  }
 
  function LoadVideo() {
 
    if (!XPMobileSDK.library.Connection.directStreamingServer) {
      alert('Your server does not support Live Direct Streaming!');
    } else {
      XPMobileSDK.getAllViews(function(items) {
        for (var i = 0; i < items[0].Items[0].Items[0].Items.length; i++) {
          let videoElement = document.createElement('videos-stream');
          videoElement.cameraId = items[0].Items[0].Items[0].Items[i].Id;
          videoElement.name = items[0].Items[0].Items[0].Items[i].Name;
          document.body.appendChild(videoElement);
          videoElement.dispatchEvent(new CustomEvent('start'));
          videoElement.addEventListener('beginVideoStuck', streamError);
          videoElement.addEventListener('endVideoStuck', streamError);
          videoElement.addEventListener('restartStream', streamError);
          videoElement.addEventListener('fallback', (event) => {
            let player = event.target.shadow.lastElementChild.getElementsByClassName('player')[0];
            let errorMsg = document.createElement('div');
            errorMsg.innerHTML = "Could not start direct streaming.";
            player && player.parentNode.appendChild(errorMsg);
            player && player.parentNode.removeChild(player);
            errorMsg = null;
            player = null;
          });
        }
      });
    }
  }
</script>
 
</html>

Hello Chin,

I think your problem is relates with browser restrictions not with the SDK itself. This is the general restrictions to autoplay a video:

Autoplay availability

As a general rule, you can assume that media will be allowed to autoplay only if at least one of the following is true:

  • The audio is muted or its volume is set to 0
  • The user has interacted with the site (by clicking, tapping, pressing keys, etc.)
  • If the site has been whitelisted; this may happen either automatically if the browser determines that the user engages with media frequently, or manually through preferences or other user interface features
  • If the autoplay feature policy is used to grant autoplay support to an

Hi Teodor,

Changing the muted value on the video element did the trick and it was able to autoplay on page load. For anyone that wants to know how I did that:

var tVideo = streamElement.shadow.querySelector('video');
tVideo.muted = true;

Things to note about it is that the video-streams will unmute itself.

There is also 2 other things I wanted to ask pertaining to the direct streaming:

1 - Is there any documentation on the videos-stream module? I know it is pretty new and didn’t see anything in the javascript documentation for the MobileAPI. One of the things I would like to due is be able to execute a command to the videos-streams to mute/unmute since audio can be distracting.

2 - There were a lot of ‘endvideostuck’ events being sent from the videos-stream. Is this something to be concerned about, and is there anything I can do to reduce the frequency of this happening?

Hello Chin,

Thank you for sharing your solution how to mute the video-stream.

About your questions:

  1. Unfortunately there is no documentation yet as we are polishing the comportment.
  2. I have added a item in my backlog to check what is the case in endvideostuck event. You can expect any official fix as soon as next release.

Best regartds

Teodor