Audio over Open Network Bridge

MIP
, , , ,
1

Hello,

We have recently installed 2022 R3. We installed it because a previous question said that it may support audio being streamed within the same RTSP stream as video using the Open Network Bridge. However, we cannot seem to get the audio stream to wok. We have been using VLC to test things. The video plays fine, and the VLC codec information detects aduio but the audio does not stream. My questions are, 1) is audio fully supported now in 2022 R3, 2) are there any documentation explaining how to configure auido to stream over RTSP using the Open Network Bridge, 3) what audio codecs are supported, 4) does audio stream within the same URL as the video over the Open Network Bridge and 5) the Management client shows that the MP4A-LATM codec is being used for a microphone, can you explain more on this? For instance, if a camera is configured as G.711A/G.711Mu will it be changed to MP4A-LATM?

2

Hi Tawana,

Let me try to answer your questions.

  1. In 2022 R3 Audio is still supported as a beta.
  2. Therefore there is no any official documentation yet.
  3. Currently supported codecs are only G.711 (u-law) and AAC. In addition we’ve found that there is an issue in AAC transport and currently only AAC with 44.1 kHz sample rate is supported. The issue was found too late in the game, and therefore is not included in the current release. It could be made hot-fix tough.
  4. There are two options for how audio could be streamed:
  • As a separate RTSP stream, containing only audio. It uses the same pattern as those for the video streams: rtsp://<onb_address>:554/live/<mic_id>, where <mic_id> is the unique microphone identifier in the VMS. This works always.
  • As a complementary stream to the video stream of the camera, which the microphone is related to. In that case you have to use the camera address: rtsp://<onb_address>:554/live/<cam_id>. And select the needed stream(s) via RTSP protocol (SETUP command). The later works only when enabled in the ONB settings in the MC (“Enable related microphones”).
  1. The RTSP server doesn’t perform any media transcoding (neither on video, nor on audio). So if the original camera codec is G.711, it has to be transported out of the RTSP server as the same - G.711. The setting you are referring to determines what would be the transport of the audio (only) if the original stream (camera) codec is AAC. And in that case you could choice between MP4-LATM and mpeg4-generic (again, only as a transport for the AAC payload).

Some additional toughs:

Similarly to video streams, ONB tries to detect the codec type from the device driver settings.

If that fails (not such property for example), is made try (audio) codec to be recognized as a part of the microphone name.

Last resort is the default setting in the MC (right after “Enable related microphones”).

So you have to be careful to configure those if audio type is not immediately recognized.

In your particular case I would advice you:

  • Determine what is the audio codec type in the device driver settings for the Microphone in the MC (Management Client).
  • Check what is reported by the RTSP steam codec type (in the VLC as you mentioned or directly in the Wireshark on the Describe command response)
  • If there is mismatch between those two, try to include the codec type in the microphone name (ex. “Microphone 1 - G.711”) or change the default audio codec type in the ONB settings section in the MC.
  • If the camera codec is set to AAC, try to choice option with 44.1 kHz sample rate (if available).
  • Try to access the audio directly on the audio address, specifying the microphone id. We have noticed some situation where in combined streams (audio and video together) one of the components (media) is not played due to some timing issues between streams.
3

Hi Petar,

Thanks for your response. In doing more testing, we discovered where the issue we was having was coming from. We have several different cameras (AXIS, HIK, Amcrest, etc.) we are testing with and what we found is that when adding an AXIS or HIK camera, which uses their own respective drivers when being added, R3 adds the audio codec that’s set on the camera. However, when adding an Amcrest camera (note the audio codec on the camera is set to G.711) using the ONVIF driver, the audio codec is being changed on the camera to AAC before being added to Milestone and once its added to Milestoine, it shows up as MP4-LATM in Milestone. I know Amcrest is a lower model camera but we using all kinds to test with. Also, Amcrest is the only camera we saw that had this issue when using the ONVIF driver. Also, note that when we tested adding the same camera to our 2022 R1 system, the the audio codec is not being changed on the camera and is being added to Milestone showing the correct G.711 codec. So for now we are not using Amcrest cameras in our audio testing.

We are now trying to get audio working for archive video but it does not seem to work. Does R3 support archive audio? Also, I know that you said that R3 was a beta version for audio, do you know when, and which version, audio will be officially supported?

4

Hi Twana,

I believe audio is not implemented in the ONB for recorded steams. I will ask the PO what is the plan for the official release and what is the status of recorded steams.

5

Hi Twana,

As far as I understood from the PO, Live audio support in the ONB will be officially released in 2023 R1. As for the Playback (recorded) one - there is still no strict plan for that. In best case it could be 2023 R2. But I won’t bet for that.

6

Hi Petar,

Can you provide a release date for the 2023 R1 release? We are planning our release and wanted to see if 2023 R1 would be released in time for use to certify against it. Thanks

7

Hi Tawana,

The GA of 23 R1 should be in first half of March (if everything goes smooth). Lets say something around 8th to 10th.

But of course this is an estimation, not a commitment :slight_smile:

8

Hi Tawana,

Can you send us Wireshark trace of adding Amcrest camera and audio codec change so we can investigate why this problem occurs?

Regards,

9

Hi Kalin,

Sorry for the delay in responding. I had to get approval to send you the Wireshark trace. Can you proivde an email address where I can send the file? Thanks

10

Hi Tawana,

My e-mail is kis@milestone.dk.

Regards,

11

Hi Tawana,

Regarding the issue you’re experiencing with Amcrest cameras and audio codec changes when using the ONVIF driver it’s not uncommon for different camera models to behave differently, due to variations in implementation and compatibility.

In terms of the audio codec, MP4-LATM is essentially AAC packaged within an MP4 container. LATM is a method of transporting AAC audio streams. So, technically, there’s no difference between MP4-LATM and AAC; MP4-LATM is just a specific format of AAC within an MP4 container. This conversion might be a normal behavior enforced by the camera or the VMS during the streaming process.

Feel free to let me know if you need further clarification or assistance.