We are currently working on integrating our camera driver based on MIPS Driver Framework with the Milestone system and have some clarifications regarding the speaker session.
During the addition of cameras in the XProtect Management Client, multiple sessions are created via MIP Driver Framework StreamManager class, including VideoStreamSession, MetadataStreamSession, MicrophoneSession, and SpeakerSession.
In these sessions, while the Video, Metadata, and Microphone sessions transfer the data from camera driver to the Milestone recording server, but the SpeakerSession is expected to receive data from Milestone recording server.
Since the speaker session is invoked via StreamManager from Milestone recording server , we are returning null pointer for the speaker session, this results in continuous re-creation of speaker session. Please refer below sequence flow diagram in Management client for IP Speaker addition. Could you please help us to understand what is the intent of creating speaker session via StreamManager from Milestone recording server as Stream Manager invokes GetLiveFrame continuously.
We noticed, if a user talks from the XProtect Smart Client, Milestone recording server creates speaker stream via speaker manger. Then it sends the audio talk data through the SendFrame API to our Camera Driver which in turn delivers the packet to IP camera. Please refer below sequence flow diagram in XProtect Smart Client for IP Speaker Talk and Release feature. Could you please confirm if below understanding is correct?
The purpose of the SpeakerSession is for the recording server to receive the data actually provided by the driver to the speaker, in order to potentially record that data in the media database. To that end, returning null is understood by the recording server as a failure to create the session, and thus it will retry continuously until a session is successfully created.
The demo driver demonstrates what is expected: A speaker session should store the latest data packet sent to the device, and return it on a call to GetLiveFrame, then clear its cache.
Here Speaker session is created by Recording Server via MIPS Driver Framework when the camera is discovered in Milestone Management Client and is added to the Milestone Recording Server. The camera supports video sensor, Microphone, Speaker, Meta data, Alarm Input, Alam Output. When the camera is added in Management Client, no user is speaking to send the audio data to the camera via camera driver. Only in Milestone Smart client, user can click Talk button and speak and then audio packets sent to camera driver via driver framework, camera driver delivers the audio to the camera device. In Smart Client case, camera driver has audio packets and can provide to GetLiveFrame API , incase of Management Client it does not have any audio packets, so the camera driver not able to provide audio packets to Recording Server via GetLiveFrame API.
Even if you have no data to provide, the MIP Driver Framework still needs to be able to create a session. In that case, calls to GetLiveFrame should return false, and the out parameters be null, but the Session still needs to be created on request. Returning null on CreateSession is a failure state and should not happen.
We tried your suggestion related to Speaker Session today.
We created speaker session and return the pointer to mips driver frame work when speaker is added. Then GetLiveFrameInternal() API invoked by driver frame work, since we do not have any audio frame, we return null for out params i.e header, data and the api return value is false. GetLiveFrameInternal() API called for next 10 seconds. After 10 seconds, mips driver frame work calls Close() API to close the existing speaker session and calls CreateSession() API after 10 seconds. This process repeats for every 10seconds.
CreateSession----->GetLive Frame(10 seconds)-> Close → 10 seconds interval ->CreateSession.
Could you please clarify any specific reason Close() API called after 10 seconds in above case.
I don’t think there is anything unusual going on here. The recording server is creating a session, after 10 seconds of having no data it decides that there is probably no activity at this time, it then waits 10 seconds and tries again. I believe the idea is to not overload the driver when there is no data, while still not waiting for too long if data should be available.
Every time Create Session() called , camera driver creates RTSP Session with the camera and setup audio back channel track and waiting for speaker data in RTSP Session, since there is no speaker data, driver frame work calls Close() API which internally close RTSP Session with the camera. The creation of RTSP session and close for every 10 seconds is put load on our camera.