How to send data to Video Channel?

Hi all,

Sorry for frequently posting and asking questions. I just have some questions related to video push.

  1. How do we send data to the Video Channel? I’ve seen the expected data in the documentation but I’m not really sure how it is implemented.
  2. How do I send JPEG file to the Video channel?
  3. Also, according to the documentation, “Video channel is implemented with private binary protocol”, what does this mean?

I have already performed all steps prior to Video Push (e.g. connect, login, request stream (both pull and upload) and have successfully received OK responses) but am now stuck in sending JPEG data through video channel.

Additional info: I am implementing this through Java programming language and I’m using Essential+ 2018R1.

Thank you.

Hi Dulce,

Your current steps (up to here) are pretty much okay.

I’ll try to draft what is left to be done (Assuming Connect, Login and RequestUploadStream passed okay).

  1. Construct upload URI from server address, fixed part and StreamId (or VideoId) received from the RequestUploadStream response.

For example:

  • Server address: 192.168.0.1:8081
  • FixedPart: XProtectMobile/Video
  • SrteamId: 3AFF3C41-798E-4D55-8C17-7D5F081B3FFE

Resulting address for video push channel:

http://192.168.0.1:8081/XProtectMobile/Video/3AFF3C41-798E-4D55-8C17-7D5F081B3FFE

2. Open HTTP(S) request to the resulting address.

3. Write/send correct Video channel “Main header”

(described at “http://doc.developer.milestonesys.com/mipsdkmobile/”, Getting started…–>Protocols–>Mobile Server protocol–>Video Channel)

The fields that should have valid values are at least:

  • Video ID - the same ID received from “RequestUploadStream” response, but in binary form and with reversed Indians.
  • Frame Size Bytes - size of the actual JPEG frame that will be send, with reversed Indians
  • HeaderSizeBytes - size of the header. In this case 36. Again with reversed Indians.

4. Write send JPEG frame binary data.

5. Close the HTTP request.

6. Repeat the steps from 2 to 5 for the next JPEG frame.

Step 3 is the most tricky one.

The actual binary protocol is the same as those for getting the video from the Mobile Server.

You could use it to sniff/reverse engineer/understand better the binary protocol for video/jpeg frames.

Unfortunately Video Push is included only in .NET and Web SDKs, but not in Android and iOS ones.

Good luck !

Hi @Dulce Polinar​,

Here is a sample code on Java for Video Push in application that uses MIP SDK Mobile. The steps that should be followed are:

1. Connect to Milestone Mobile Server.

2. Login to the server.

3. Request a video push channel.

4. Push all the frames that you want with an incremented frame number.

5. Call stopVideoStream command from the MIP SDK Mobile to close the opened channel.

And the code is:

private static String videoChannelID = “”;

private void requestNewPushChannel(){

xpMobileSDK.requestVideoStream(null, null, null, null, CommunicationCommand.PARAM_SIGNAL_UPLOAD, CommunicationCommand.PARAM_METHOD_PULL, null, new SuccessCallback() {

   @Override

   public void onSuccess(CommunicationCommand response) {

       _videoChannelID_ \= response.getOutputParam().get("VideoId");

       //After the received video push channel ID, you could start sending frames to the Mobile server.

   }

}, new ErrorCallback() {

   @Override

   public void onErrorOccurred(CommunicationCommand cmd) {

       //Error occurred when requestVideoStream was called.

   }

});

}

/**

* Pushes frames to the Mobile server. Before sending your frames to the Mobile server, you should request a new video channel id. Then you can send all the frames by calling pushFrame method. When you want to stop sending frames, you should stop calling pushFrame method and call stopVideoStream command from the Mobile SDK.

* @param frameAsByteArray - the frame that will be pushed to the Mobile Server. It should be in JPEG format.

* @param frameNumber - number of the frame. Every next should be incremented with 1

*/

private void pushFrame(final byte[] frameAsByteArray,int frameNumber) {

if (videoChannelID != null && !videoChannelID.equals(“”)) {

   String videoAlias = new StringBuilder("/").append(_communicationAlias_).append("/Video/").append(_videoChannelID_).append('/').toString();

   try {

       HTTPConnection conn = new HTTPConnection(address, port, videoAlias);

       ByteArrayOutputStream bOut = new ByteArrayOutputStream();

       byte\[\] hBuffer = new byte\[36\];

       bOut.write(hBuffer);

       bOut.write(frameAsByteArray);

       VideoCommand vcmd = new VideoCommand(_videoChannelID_);

       byte\[\] frameBuffer = bOut.toByteArray();

       vcmd.refactorMainHeader(frameBuffer, frameNumber, System._currentTimeMillis_());

       conn.sendByteArrayRequest(frameBuffer);

   } catch (IOException e) {

       //Handle the IOException ...

   }

}

}

Thanks for both of your help, @Petar Vutov​ and @Veselin Yordanov Petev​. I have finally managed to send my JPEG images to the server. Cheers!

@Petar Vutov@Veselin Yordanov Petev

I’m wondering, why I need to establish an HTTP connection for every frame?

In my case i want to implement a Live Streaming demo app with start and stop button, when the user clicks start it continually takes photos and send them over the connection and when user clicks stop it stops sending frames?

wouldn’t it be inefficient to have HTTP connection created for each frame?

please help.

Hi Mostafa,

Generally speaking you are right.

Passing all the JPEG images into single HTTP request could be more efficient from network utilization perspective.

Unfortunately Mobile server doesn’t support it.

Thank you for your fast response,

As i mentioned i’m trying to build a live streaming demo app, and I used the Milestone’s official app and the video push works pretty well, however when i use the code above the streaming has two issues:

1)There is a HUGE lag between frames and it is shown as a distinct images instead of a continues video.

2)when I’m trying to view the video being streamed by the demo app using the official milestonen’s app it shows a connection lost message for a short period of time then it shows a frames then another connection lost message and so on.

So how did you guys managed to have this smooth streaming functionality in the official milestone’s app?

what android Camera API did you used?

Hello,

The connection lost message you receive is probably due to the very big gap between the frames. If we do not receive a frame for a few seconds, we assume the connection is broken.

To increase the fps, you may want to use lower camera resolution, and high JPEG compression.

We just use the standard Android Camera API.

Hope this helps

There are multiple standard Android Camera APIs →

  1. Camera API

  2. camera2

  3. CameraX

and I’m already using the least resolution. however it takes time to take a jpeg photo which effects the overall performance and the resulting stream.

thanks your your help

Yes, 1) Camera API is what we use at the moment.

I’m not sure why it takes so long to take a jpeg photo, normally it should be quite fast…

@Plamen Petkov Parvanov (Milestone Systems)​ @Petar Vutov

Thanks guys, I managed to have the live streaming functionality up and running, the problem was that i used CameraX api to have the raw camera frames and i had to change the encoding twice to get it work YUV420_888 ->NV21 → JPEG

and this cause the lag.

but i’m having an issue that at some point while streaming some frames get out of order and a frame is shown repeteadly then it continue works fine

i tried to reduce the FPS by using the following camera configurations :

// params.setPreviewFpsRange(1000, 5000)

//JPEG quality = 50

it worth noting that after I added the following line

Thread.sleep(250)

before

vcmd.refactorMainHeader(frameBuffer, frameNumber, System.currentTimeMillis())

conn.sendByteArrayRequest(frameBuffer)

this out of order frames behaviour reduced,but the frames dropped significantly.

Any input regarding this issue?

Hi Mostafa,

Seems to me like async/threading issue.

Could you ensure that you try to send next video frame only when previous one was already sent ?

Otherwise making multiple POST requests simultaneously could lead to such behavior.

As http layer could not guarantee any order of receive of the requests.