Is there an introduction to using XML messages in the Mobile Server Protocol?

MIP
, , ,
21

An update to the above post…

I’ve checked what I’m using in Java and it produces the same output for the example in the other thread, as you describe in your post that starts “I’ve tested this sequence of operations on the server.” (Sorry, I can’t find a way to link to the actual post - there doesn’t seem to be an anchor that I can use.)

I get the same output as you, so it looks like the libraries I’m using agree with what you describe needs to be done, but when I’m using the same process in my code, I still get an error 19. Any thoughts why that might be?

22

As you already checked the algorithm, the only thing that comes to my mind is the conversion of the common shared secret key to Hex string. It should have exactly 256 bytes. You should remove leading / trailing zeros if such exist (some implementations of Big Integer do that). And you could try to reverse the order of the bytes. For example in the .NET code the byte[0] of the key is the first 2 characters of the Hex representation. But depending on the platform it could be the opposite.

And just to repeat it again - the key used is Common shared secret key. Neither the private one, nor the public one.

23

Thanks - good tip! I’ll check how long the key is. I’m pretty sure it’s the shared key I’ve used - but I will double-check!

And when you say byte order, I assume you mean big-endian vs. little-endian?

24

Well,

common shared secret key is usually presented as some byte array in the memory.

In that perspective its conversion to HEX string could be made in at least two different ways - first byte to be presented as first two characters in the resulted HEX string or vice versa - last two characters. Similar to Endians for WORD and DWORD representations.

If however the in-memory representation of the key is in other form - like array of integers, yes endians (for the particular data type) could need to be corrected.

25

Ah OK! I didn’t realise the byte array might need to be reversed. I’ll try that with both big- and little-endian bytes.

EDIT: Tried that now, but always with the same result. (So I’ve now tried four combinations: big- and little-endian combined with reversed/not-reversed.)

Also, the key is definitely 256 bytes not 256 bits? Mine seems to be the latter…

EDIT: Yes, my shared key is definitely 256 bits not 256 bytes: there’s private static final int KEY_SIZE = 32; in PBE.java. I’m now wondering if it could be something to do with how the keyBytes passed to the PBE constructor are split between the IV (I’m not sure what that actually is) and the Key.

EDIT 2: I’m pretty sure the splitting is done correctly since if I alter how that’s done, the username/password encryption fails. So I’m confident the 256-bit key is correct now. It’s just how to correctly convert it to hex that seems to elude me.

26

Aah,

I think I 've got what the problem is.

You are using the Key and IV (initialization vector) that are used for initialization of the AES-256 encryption.

I’m referring the whole common shared key that is received with the ModPow operation of your private key and the server’s one public key.

And it is quite longer. From this long shared secret key are used only 48 bytes in order to be initialized PBE - 32 for the Key and 16 for the IV.

Now I realize that provided code (crypto.zip) doesn’t have support for that.

Let me make few changes and I’ll apply an new version.

You could compare with the old one and see what the differences are - e.g. what to use in order to get the full size of the common shared secret key.

27

Thanks - looking forward to trying the new version.

28

I’ve added extensive comments with directions what to try, as I haven’t tested the code.

To be honest I didn’t event compile it, because Xamarin tools ruined my local Java build.

But you will got the point.

29

Thanks - but sorry to hear your Java build broke as a result!

I’ll have a read of the code and let you know how I get on.

EDIT: Yes! That worked! Error 19 is now gone and I get an OK result! Thanks very much - that is excellent progress. :grinning_face: Now to complete the sequence of commands (tomorrow). FYI, it looks like my key length is 128 bytes.

EDIT 2: Interestingly, it worked the first time but now I keep getting a key length of 129 bytes, which I think it doesn’t like. I’ll investigate tomorrow…

EDIT 3: When the length of the key is 129 bytes, which it is every so often, removing the first byte seems to fix it. I think this ties in with your comments in the GetFullSecretKeyBytes method.

30

Next question… :slightly_smiling_face:

I’ve been looking at GetSeqences on this page. The example given looks like this:

<?xml version="1.0" encoding="utf-8"?>
<Communication
    xmlns:xsd="http://www.w3.org/2001/XMLSchema"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <ConnectionId>3a2fcbae-d387-441b-9be4-2020b9731b43</ConnectionId>
    <Command SequenceId="79">
        <Type>Request</Type>
        <Name>GetSequences</Name>
        <InputParams>
            <Param Name="ItemId"   Value="230ae9f1-17bc-4c3b-806f-9fdf048fe760"/>
            <Param Name="SeqType"    Value="Recording"/>
            <Param Name="Time"       Value="0"/>
            <Param Name="AfterTime"  Value="1358258893"/>
            <Param Name="AfterCount" Value="1"/>
        </InputParams>
        <OutputParams/>
    </Command>
</Communication>

Based upon that, I’ve made this request:

<?xml version="1.0" encoding="utf-8"?>
<Communication xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <ConnectionId>5a535d7b-eb49-466c-96c2-fbc40521ec58</ConnectionId>
  <Command SequenceId="3">
    <Type>Request</Type>
    <Name>GetSequences</Name>
    <InputParams>
      <Param Name="SeqType" Value="Recording"/>
      <Param Name="ChalAnswer" Value="aVWc3hDu1WTmQBxpYAxZuTDh20zhn8O7D4WKpU2PNLm4b/BsH4ZnnAhYK+9fhQubHlBfGDyqs+oPJk9QQP73/g=="/>
      <Param Name="AfterCount" Value="100"/>
      <Param Name="Time" Value="0"/>
      <Param Name="Challenge" Value="1B9FE4A7-E1DA-4396-8608-E5CE08B618DC"/>
      <Param Name="CameraId" Value="bd45dcdb-d1f0-4b73-8532-d78ba2fdf873"/>
      <Param Name="AfterTime" Value="1624012649"/>
    </InputParams>
  </Command>
</Communication>

So Time is 1/1/1970 and AfterTime is the number of Epoch seconds for “right now”. This returned only the oldest recordings.

Important: One thing I did have to do was change ItemId to CameraId because I got an error back when I used ItemId telling me CameraId was missing from the request parameters. Looks like the documentation is slightly out of date.

I would expect that:

<?xml version="1.0" encoding="utf-8"?>
<Communication xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <ConnectionId>5a535d7b-eb49-466c-96c2-fbc40521ec58</ConnectionId>
  <Command SequenceId="3">
    <Type>Request</Type>
    <Name>GetSequences</Name>
    <InputParams>
      <Param Name="SeqType" Value="Recording"/>
      <Param Name="ChalAnswer" Value="aVWc3hDu1WTmQBxpYAxZuTDh20zhn8O7D4WKpU2PNLm4b/BsH4ZnnAhYK+9fhQubHlBfGDyqs+oPJk9QQP73/g=="/>
      <Param Name="BeforeCount" Value="100"/>
      <Param Name="Time" Value="1624012649"/>
      <Param Name="Challenge" Value="1B9FE4A7-E1DA-4396-8608-E5CE08B618DC"/>
      <Param Name="CameraId" Value="bd45dcdb-d1f0-4b73-8532-d78ba2fdf873"/>
      <Param Name="BeforeTime" Value="1624012649"/>
    </InputParams>
  </Command>
</Communication>

(which is exactly the same thing but using BeforeTime and BeforeCount instead of AfterTime and AfterCount) would return exactly the same thing, or possibly (ideally!) the most recent 100 recordings, but it returns an empty set of sequences.

Am I misunderstanding the API or does it not work as described…?

EDIT:

I have tried setting now to the Epoch seconds and aMonth to 2678400 (which is 60*60*24*31), as the day for which the earlier request returned sequences was less than a month ago. I used the exactly same approach, but instead of:

<Param Name="Time" Value="0"/>
<Param Name="AfterTime" Value="<now>"/>

I had:

<Param Name="Time" Value="<now - aMonth>"/>
<Param Name="AfterTime" Value="<aMonth>"/>

where the angle bracket text would obviously be replaced by real numbers.

It returned no sequences at all, despite including the period that definitely has sequences!

31

Hi Richard,

I think you’ve misunderstood the documentation.

I’ll citate the most interesting parts of the parameters of GetSequences:

Time - Central time of sequence request (in milliseconds since Unix epoch).

BeforeTime - (optional) Time duration in the past (relative to the central time) to where sequences will be searched (in seconds, not seconds since Unix epoch). Must be non-negative.

AfterTime - (optional) Time duration in the future (relative to the central time) to where sequences will be searched (in seconds, not seconds since Unix epoch). Must be non-negative.

In your first request:

      <Param Name="Time" Value="0"/>
      <Param Name="AfterTime" Value="1624012649"/>

You really asks for sequences starting from 1st of Jan 1970 till “now”.

But in the second:

      <Param Name="Time" Value="1624012649"/>
      <Param Name="BeforeTime" Value="1624012649"/>

your “Time” parameter is not correct. It should be in milliseconds, not in seconds. So you actually provided Time == 19th of Jan 1970.

In result you’ve requested sequences from 19th Jan 1970 to around somewhere in 1919.

You have to change that “Time” to 1624012649000.

Tip: Your strategy of getting most recent sequences is good. You could however move the “Time” a little bit in the future in order to ensure any clock differences between servers are taken into account and you’ve got the last sequence, if it is under creation right now. I’m having in mind especially the case where “Time” == .

In the last example :

<Param Name="Time" Value="<now - aMonth>"/>
<Param Name="AfterTime" Value="<aMonth>"/>

I would assume you provided now again in seconds and not in milliseconds, so basically you’ve requested sequences from 19th Jan 1970 to 19 Feb 1970.

32

You’re quite right - thanks. I guess I expected all times to be in seconds or all in milliseconds!

Can I assume that it returns sequences that are the closest to Time?

Thanks for the tip about the edge condition.

33

Yep, It should return the sequences closer to the “Time” parameter.

34

As for the CameraId and ItemId parameters - what version of the Mobile Server are you using ?

35

I guess it’s Corporate 2020 R3.

36

I’m getting really close now, but I’ve started getting a weird error.

If I do a sequence such as the following…

  • Connect
  • LogIn
  • GetSequences
  • GetSequences
  • GetSequences…

…then at some point (within a minute or two, usually) it will decide that the connection ID I’m using is no longer valid and returns an error 12. I’ve no idea why it would suddenly become invalid after such a short time.

I’ve started catching that error and reconnecting when it happens. Then the sequence continues:

  • Connect
  • LogIn (which is successful)
  • GetSequences (which returns error 19)

I’ve made sure I clear the list of challenges when I reconnect, so I don’t understand why GetSequences fails when the LogIn succeeded.

Of course, since I am reconnecting, I also have to create new keys, but this must have worked if LogIn succeeds, right? I’ve looked at the DHEncryption class you provided, and I don’t see anything static in there that might be left over from the previous instance either.

Any thoughts as to why this might be happening…?

EDIT:

I may have solved the latter part of this. :slightly_smiling_face: I was reusing a challenge and challenge answer when I re-made the failed request; now I get new ones and it works. I still don’t know why the connection ID becomes invalid so quickly though.

Also, if I reconnect, do I really have to generate new keys? Or can I somehow reuse the existing ones? Can I maybe reconnect with the same connection ID??

EDIT 2:

Is this perhaps something to do with the PeriodicLiveMessage input parameter and/or the Timeout output parameter? I can’t find any reference to a “Live message” on the API documentation page, but I remember seeing such things from the VideoExport sample. Looking at the request and response, I am wondering if I need to send such a message at least every 30 seconds (since the timeout seems to be 30s)?

EDIT 3:

I’ve added a timer that makes a LiveMessage request every 25 seconds (given that the timeout is 30s) and so far this seems to have worked - the connection stays valid. I’m not sure what PeriodicLiveMessage is, however, since the documentation says this defaults to “No” and I haven’t changed it to “Yes”.

37

Seems you figured it out :slight_smile:

Mobile Server (MoS) has feature called “WatchDog”. It requires clients to “ping” it in order to keep the “session” alive. As you’ve written “Timeout” parameter defines the maximal interval on which client should ping the server. And as you found out it happens trough the “LiveMessage” command.

The default value of the timeout is 30 seconds, but could be changed from the MoS configuration. You should calculate dynamically how often to send Live messages, based on the Timeout value. For example (2/3)*Timeout.

Unfortunately “PeriodicLiveMessage” has left in the documentation since previous versions and is not active right now.

Meaning you cannot turn off watch dog functionality of the MoS.

38

Thanks Petar, that’s just what I’ve done. I use Timeout-5 instead of Timeout*2/3 which might be a bit risky! :grinning_face_with_smiling_eyes:

39

I had to mention that valid values for the Timeout in the MoS configuration are from 5 to 300 seconds.

Using such lower than 25-30 seconds is not something Milestone usually recommends.

40

Thanks, I’ve increased my server’s timeout now.