Is there an introduction to using XML messages in the Mobile Server Protocol?

I’ve been looking at this page: https://doc.developer.milestonesys.com/mipsdkmobile/index.html?base=reference/protocols/mobile_command.html&tree=tree_5.html

It seems pretty comprehensive about what can be done, but doesn’t seem to give enough information to actually use the protocol.

There are a few things that aren’t clear to me. When connecting, for example, what format should the client’s public key be in? Can I make up my own client ID? Is it like the Image Server API, where you can (must) keep the connection open, or is it one connection per request? (On this page it says “The client always initiates HTTP request which contains exactly one XML message…” but on this page it says “There is a commands separator (separator between different XMLs send on the link), which is two new lines (“\r\n\r\n”). It should be possible for the client to send multiple commands before to retrieve any server response.” which seems to disagree.)

The intro page has a communication flow diagram, which is helpful but again doesn’t provide enough information to actually use the protocol.

Is there somewhere that explains exactly what each parameter is (including its format and, where appropriate, where it comes from) and ideally gives example message flows using the XML messaging?

EDIT: A bit of context… What I would ultimately like to do is use the XML protocol to export and download [untranscoded] clips of video from a specified camera. It sounds like I can connect, log in, and trigger the export (although I now see that the StartExport command is obsolete!), I haven’t worked out how to get the exported video yet. So ideally, I would also like an example of using the XML protocol to do all that!

Hi Richard,

You are right. The information provided is not complete in order to easily implement the protocol by yourself.

I would assume you are using platform that is not in the supported ones by the MIP SDK Mobile. If that is not the case I’ll highly encourage you to try to use one of the SDKs.

I’ll try to address your questions bellow:

Q: When connecting, for example, what format should the client’s public key be in?

A: Client public key is in form of base64 encoded string.

Advice: Making DHKE work is a hard task on it’s own. In the beginning you could try to skip that. For this to happen you should not provide PublicKey on the Connect command. In addition have to be made change in the Mobile server configuration file. Please open it and find “” tag. In it, right after “” add:

    <Credentials>
      <add key="PlainTextAuthenticationEnabled" value="True"/>
    </Credentials>

Using plain text credentials could be dangerous and is by default prohibited. But for debugging purposes, as well as if your Mobile server is running on HTTPS it could be quite handy.

Q:Can I make up my own client ID?

A: No. ConnectionId is generated by the server in the response of the Connect command. So basically the first command you send is Connect. And for it you do not specify ConnectionId. On the response of the command you receive (in the XML data) connectionId, that is used for all the subsequent commands to the MoS (Mobile Server).

Q: Is it like the Image Server API, where you can (must) keep the connection open, or is it one connection per request?

A: No. The MoS protocol is HTTP based. So you should comply with the HTTP standard. If you are using HTTP/1.1 you can use different TCP connections for every HTTP request. Or you could share same TCP connection, as well as you wait for the response from the previous request first. If you are using HTTP/2.0 you could have multiple HTTP request one after another, before to receive the first HTTP response.

Q: “The client always initiates HTTP request which contains exactly one XML message…” but on this page it says “There is a commands separator (separator between different XMLs send on the link), which is two new lines (“\r\n\r\n”). It should be possible for the client to send multiple commands before to retrieve any server response.” which seems to disagree.

A: In it’s initial form and release MoS Protocol supported possibility multiple commands to be send one after another in a single request. This however seemed to introduce more troubles than to solve actual problems. So now this strategy is abandoned and is not supported. The message separator (unfortunately) left from this times.

So now there is a clear 1:1 relation between commands and HTTP request.

Q: The intro page has a communication flow diagram, which is helpful but again doesn’t provide enough information to actually use the protocol.

A: Well the actual session could vary depending on what the actual integration would be. The common thing between all the sessions is the order of the commands. It should be:

Connect

Login

<any_other_command>

Disconnect

So you should always start with Connect and Login after that. And call Disconnect when you finished your work with the MoS.

Q: Is there somewhere that explains exactly what each parameter is (including its format and, where appropriate, where it comes from) and ideally gives example message flows using the XML messaging?

A: I’m not sure we have such detailed information on the protocol level. It would be quite difficult to support such.

Q: What I would ultimately like to do is use the XML protocol to export and download [untranscoded] clips of video from a specified camera. It sounds like I can connect, log in, and trigger the export (although I now see that the StartExport command is obsolete!), I haven’t worked out how to get the exported video yet. So ideally, I would also like an example of using the XML protocol to do all that!

A: Well, I’m not sure we could/would provide such an example. What I could advice you is to try to use one of the Web Samples and look at the traffic in the browsers Network tab. If I remember well we do have “VideoExport” sample, which is something very close to what you want to achieve.

As for the “StartExport” command - yes, it is really an obsolete one. It is still supported by the MoS, but could be removed in future versions.

In the response of the “StartExport” command you get “ExportId”. This Id could be further used with “GetExport” command. It will return the status of the export (0 to 101) as well as (relative) link for a download.

Please bear in mind that in future version download could require call of yet another command.

In general export commands are considered as obsolete as they are replaced by Investigations commands. In short you should first “CreateInvestigation” and later call “StartInvestigationExport”. Investigations are in general more feature rich and also support audio. Otherwise both Investigation exports and obsolete (old) exports support the same export formats - XProtect (DB), AVI and MKV. I would suppose you are targeting MKV exports.

Hi Petar,

Thanks for the information - that has been really helpful.

I’m using Java to access the APIs, which so far has been quite successful. It seems the SDKs are all Windows-based and we don’t want to be tied to that.

A few comments on my questions and your answers…

  1. I did guess that the public key is base64-encoded, but base64-encoded what? What format key has been base64-encoded?
  2. When I referred to the client ID, I meant the device ID (the “DeviceId” input parameter on connect) not the Connection ID.
  3. Regarding the InvestigationExport functionality, that sounds like something I can use. I couldn’t see a command that actually retrieves the exported video though - I thought maybe GetInvestigation did that, but it seems to get XML not video. And StartInvestigationExport seems to start exporting but doesn’t return the video.

1. The code as it is in the .NET SDK:

        public string GetPublicKey()
        {
            var publicKey = _keypair.GetPublicKey();
            var bytearray = publicKey.ToByteArray();
            return Convert.ToBase64String(bytearray);
        }

The devil is in the details when you implement DHKE. I’ll ask around if it is possible to extract few classes from the Android SDK. In most cases they should be compliable in regular Java. I’m not sure however if I’m allowed to.

You have seen the most detailed information we have is here.

2.You can omit DeviceId. It is intended for usage with Mobile devices and only if they are subscribed to push notifications.

3.No matter if you are using StartExport or StartInvestigationExport, you will receive ExportId. This Id could be further used with “GetExport” command. It will return the status of the export (0 to 101) as well as (relative) link for a download.

<?xml version="1.0" encoding="utf-8"?>
<Communication
    xmlns:xsd="http://www.w3.org/2001/XMLSchema"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <ConnectionId>7fdc93c5-d1f6-4f17-acc3-a332e898a778</ConnectionId>
    <Command SequenceId="49">
        <Type>Response</Type>
        <Name>GetExport</Name>
        <InputParams/>
        <OutputParams/>
        <Exports>
            <Export CameraId="25c9a2d9-d044-4d1e-bb74-eb7e0ae9eb05" 
...
                                         Link="XProtectMobile/Export/virtualtest/b146c13f-496f-40d9-a2c9-798f38f59673/Axis Q7404 Video Server (10.4.13.6) - Camera 1.Avi"
...
                                         User="VirtualTest"/>
        </Exports>
        <Result>OK</Result>
    </Command>
</Communication>
  1. Thanks for that, and the link especially - I hadn’t come across that before.
  2. OK.
  3. Good to know.

PS - I forgot to ask, where can I find/download the Web Samples?

Web samples could be found in the mobile server as well as i part of the MIP SDK Mobile.

You can download the MIP SDK Mobile from here:

https://download.milestonesys.com/MobileSDK/

Thanks. They weren’t in my “XProtect Mobile Server” directory in Program Files, so I’ve downloaded them. I’ve found the VideoExport sample you mentioned, so hopefully that’ll help.

I feel a bit foolish to ask, but how do I actually use the web samples? I’ve extracted the .zip file and I navigate to and double click the videoExportSample.html file but all I get is a blank page in my browser. There doesn’t seem to be a read-me document or anything like that in the archive.

EDIT: I’ve since found this page but it doesn’t quite tie up with what I’m seeing on my system, and when I visit the page on the webserver it loads (so I must have the path right) but wants to save it not actually display it! It actually gives me the option to save it or open the HTML file in my default application which is the browser itself - but if I do that the page is blank like before.

By default samples are “disabled” mostly for security reasons. In order to “enable” a particular sample, you should go in the sample folder and delete/rename the .download file. It instructs mobile server to serve the sample not as page but as a downloadable content.

As you probably already find out, the easiest way to use the (web) samples is directly form the Mobile Server installation folder (Program Files or Program Files x86). They should present under “Web\XPMobileSDK\Samples” subfolder. And you could navigate to them directly in your browser, starting with Mobile Server address and using relative path (excluding “Web”).

OK, I got the Video Export sample working. I’ve used Wireshark to see what messages are sent and received and some of it makes sense. :slightly_smiling_face:

The sequence I observed was:

  1. Connect
  2. LogIn (which returns a list of 100 “Challenges” each of which is a UUID)
  3. GetAllViewsAndCameras
  4. RequestChallenges (gets another 100 “Challenges”, none of which is ever used)
  5. RequestChallenges (yes, same again!)
  6. LiveMessage (I think this is a heartbeat of some kind)
  7. LiveMessage
  8. GetThumbnailByTime
  9. StartExport
  10. GetExport (returns, amongst other things, State=“0”, which I think is the percentage complete; CompletedTime is negative)
  11. GetExport (State is now 30)
  12. GetExport (State is now 66)
  13. GetExport (State is now 101; CompletedTime is now positive, Size is now non-zero; Link is now set)
  14. Then the download from the Link URL begins.

The challenges passed in requests after login all come from the original list. Challenges 1 to 12 are used, excluding no. 2 for some reason!

I also see that challenges are provided in responses, but it’s not clear where these come from.

Is there another page I’ve not yet found that clarifies any of this?

Mobile server supports additional “anti forgery” protection of the commands in form of chaps or challenges.

It makes sense only if your connection to the server is over plain HTTP. Otherwise it could be omitted. For this to happen you just skip parameter “NumChallenges” in the Login command, do not call RequestChallenges and do not provide Challenge on every command.

If for some reason you want to use this mechanism, please bear in mind:

  • Every command should be accompanied with a valid challenge and challenge answer (except Connect and Login)
  • Challenges are received on Login or on RequestChallenges where you specify their number with “NumChallenges” parameter
  • Single challenge is received on the response on any other command, when the mechanism is turned on
  • Challenges are generated by the server, every one of them is for one time (command, request) usage and has an expiration time of 1 hour (by default)
  • ChalAnswer is generated in the following algorithm:
    • the challenge is converted to upper case and concatenated with common shared secret key received after DHKE
    • the result (combined string) is hashed with SHA-512 and converted to Base64

From the description above you can understand that implementation of challenges is not a trivial task as you should have a valid DHKE before that and should implement active management of the challenges received by the server, in terms of count and expiration timeout.

Our SDKs provide built-in support for that in form of ChallengesManager(s), but unfortunately you cannot utilize them.

I would advice you to try to not use chaps, at least in the beginning.

Thanks for all that - it’s really helpful!

Earlier in this conversation you said: “I’ll ask around if it is possible to extract few classes from the Android SDK. In most cases they should be compliable in regular Java. I’m not sure however if I’m allowed to.” Should I assume from your SDKs comment above that this isn’t going to be possible?

I’ll have a look at enabling SSL so I can simplify things a bit!

Hi Richard,

You can find attached example implementation of the crypto part. You should use it only as a reference and without any warranty from Milestone.

Sample usage could be something like (code + pseudo code):

// Create new Diffie-Helman object to be used for the session
dhe = new DHEncryption();
 
// Get our public key (to send to Mobile Server)
String publicKey = dhe.GetPublicKey();
 
// Send Connect command to the server
// (Include our public key, and receive the server's public key in response)
serverPublicKey = doConnect( publicKey );
 
// Use the server's public key to generate the shared secret key 
dhe.GenerateSecretKey(serverPublicKey);
 
// Encrypt the user name and password with the shared key
encrypted_username = dhe.Encrypt( userName );
encrypted_password = dhe.Encrypt( password );
 
// Send Login to the server with our user name and password (encrypted)
doLogin( encrypted_username, encrypted_password );
 
// We're now connected to the mobile server!
 

Thanks for that! It has taken a while but I’ve incorporated it into my code, made a few necessary changes to make it work with “non-Android” Java and written the required extra bits to send and receive messages. Java, it seems, doesn’t have the AES/CBC/PKCS7Padding cipher, but according to https://stackoverflow.com/a/29234136/1999993 in practice, this is the same as AES/CBC/PKCS5Padding so I changed it to that. Now, however, I’m getting a rather cryptic response containing 23. Is there a list of error codes somewhere…?

List of error codes could be found here.

I cannot comment on padding, seems similar, but…

However this “written the required extra bits to send and receive messages” could be more problematic I think.

On which command you receive error code 23?

Thanks - that really helped. The error was on the LogIn command, but I realised what it was when I saw that error 23 is “Not allowed in this state” and looked again at my code. I was storing the Connection ID in what turned out to be a transient way, so it wasn’t being sent on the next request. So I’ve now successfully logged in. On to the next step…

PPS - Where’s the mobile server configuration file? I’ve looked in [C:\Program](file:C:/Program) Files and %ProgramData% but can’t find anything!

EDIT: Ah… Is it [C:\Program](file:C:/Program) Files\Milestone\XProtect Mobile Server\VideoOS.MobileServer.Service.exe.config? (I was looking for something.xml, certainly not something with a .exe.config extension!)

Yep, there it is.

I’ve run into a wall again!

I have been trying to follow another post you helped on, that one about using python, to help me calculate the ChalAnswer value. I think I’ve followed it correctly, but I keep getting an error 19 (“Security error”).

I get the challenge and append the hex version of the key from the PBE class - the key that’s defined as:

private final byte[] Key = new byte[KEY_SIZE];

I generated the hex version of the key using this code from SO (the second version).

I then convert it to uppercase and get the bytes (either using StandardCharsets.UTF_8 or StandardCharsets.US_ASCII, the error happens for both) and then I use:

MessageDigest.getInstance("STA-512").digest(challengePlusKey)

to get the hash bytes and then:

Base64.getEncoder().encodeToString(hashBytes)

to get the value for ChalAnswer. I’ve looked at the JavaScript for XPMobileSDK.library.js and that seems to agree with what I’ve done. I haven’t delved deeper into the CryptoJS class to see precisely what CryptoJS.SHA512(…) and CryptoJS.enc.Base64(…) do.

Just wondering if there’s any more in the Android SDK that might provide some clarification…?