"DecodingFailed" displaying videos in SmartClient

MIP
,
1

We are using a custom implemented driver developed with the MIP SDK and are experiencing issues displaying video footage in the SmartClient. Fetching recorded footage via the playback API, as well as live footage, results in a black screen. We have verified that footage is indeed being streamed to the Milestone system, but the client fails to display it. When enabling diagnostics in the SmartClient, it shows “Image availability: DecodingFailed”.

The environment is virtualized, so we rely on a solution without hardware acceleration. Is there any additional documentation regarding this decoding issue that could help us display the video correctly in the SmartClient?

For reference: Searching the forum, a similar issue was posted before Developer Forum

Kind Regards,
Dennis

2

It’s possible that the stream isn’t being sent in a fully compliant format, but seeing “Image availability: DecodingFailed” in Smart Client means you’re close — the stream has passed the Recording Server’s validation and is reaching the Smart Client.

Be aware that Smart Client’s hardware‑accelerated decoder is more capable than its software decoder. For example, H.265 decoding generally requires hardware acceleration. Have you tested with a Smart Client running on a system with a GPU that supports hardware‑accelerated decoding?

You can also experiment with stream parameters such as resolution, GOP structure, and avoiding B‑frames.

3

The stream we deliver is encoded with H.264, using the Main@L4.1 Profile, with no B-Frames present. I’ve attached more information about the encoding at the bottom. We also send a stream with lower settings using the Main@L3.1 Profile as our live stream, but the same issue still occurs.

Exporting the footage and viewing it on a different device works, so the export itself was successful - the SmartClient has issues displaying it depending on the hardware. We have two other virtualized setups, using different hardware setups for the virtualizer host and there the video is decoded (without hardware acceleration) correctly in the SmartClient.

The crucial thing for us is whether we can somehow guarantee that the streams we deliver through our driver are in fact usable in the SmartClient. Is there perhaps a section in the MIP SDK docs that narrows down the supported codec parameters?

Stream encoding parameters

Format : AVC
Format/Info : Advanced Video Codec
Format profile : Main@L4.1
Format settings : CABAC / 1 Ref Frames
Format settings, CABAC : Yes
Format settings, Reference frames : 1 frame
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Bit rate : 1 712 kb/s
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Color range : Full
Color primaries : BT.601 PAL
Transfer characteristics : BT.601
Matrix coefficients : BT.470 System B/G

4

Friendly reminder that this topic is still ongoing. We would highly appreciate any guidance on how to proceed.

Are there any settings in the SmartClient or Management Client we can try to get the footage decoded successfully? Additionally, are there any specific drivers or codecs required on the client side, depending on the hardware or virtualization environment?

6

Consulting a Milestone expert:
My immediate suspicion is that the MIP driver fails to add SPS/PPS NAL units in every IDR frame. That would prevent the Smart Client from attaching to the stream after it has been initiated. If that is not the reason, then check that keyframes are true IDR frames (NAL type 5).

8

Thank you for the suggestion. We verified the recorded data at every stage: the RTSP stream, the recording and the export file from Milestone SmartClient in MKV format.

The exported example is an H.264 video with a frame rate of 30 FPS and a GOP size of 30. The packets were analyzed with FFmpeg using the following command: ffmpeg -i export.mkv -c:v copy -bsf:v trace_headers -f null -. The entire detailed dump from the command is attached to the post. In short, the stream consists of a repeating sequence of a packet containing an AUD, SPS, PPS, and an IDR frame.

Access Unit Delimiter  [nal_unit_type=9: AUD (Access Unit Delimiter)]  
Sequence Parameter Set  [nal_unit_type=7: SPS (Sequence Parameter Set)]  
Picture Parameter Set  [nal_unit_type=8: PPS (Picture Parameter Set)]  
Slice Header  [nal_unit_type=5: IDR Slice (I-frame / keyframe)]

Following this, 29 packets with an AUD and a P-frame can be found. This sequence repeats until the export ends.

Access Unit Delimiter  [nal_unit_type=9: AUD (Access Unit Delimiter)]  
Slice Header  [nal_unit_type=1: Non-IDR Slice (P-frame)]

We would be glad for further suggestions.

nal.txt (8.1 MB)

9

I asked Claude to correlate the data from you with our codebase. This is the response. Note that hallucinations are possible in this type of analysis.

Based on my analysis of the forum post and the codebase, here’s my assessment for your support reply:

Diagnosis

The stream structure is correct (SPS/PPS before every IDR, proper NAL type 5, proper GOP structure). The issue is almost certainly related to two unusual VUI parameters in the H.264 SPS:

1. Color Range: Full (video_full_range_flag = 1)

This is the most likely culprit. Surveillance/broadcast H.264 streams almost universally use Limited range (16-235). Full range (0-255) is uncommon and some hardware decoders (Intel Quick Sync / Media SDK, NVIDIA NVDEC) may not handle it correctly, or they may apply incorrect color space conversion, producing visual artifacts or outright decode failures. This explains the hardware-dependent behavior — software decoding handles it fine, but hardware decoders on certain platforms fail.

2. BT.601 PAL color matrix with 1920×1080

Per H.264 spec convention, 1080p content should use BT.709 (colour_primaries=1, matrix_coefficients=1). Using BT.601 PAL (colour_primaries=5, matrix_coefficients=5) with HD resolution is non-standard. While this alone may not cause decode failure, combined with full range it increases the chance of hitting untested code paths in hardware decoders.

Recommendation for the customer

1.Change color range to Limited (video_full_range_flag = 0) — this is the standard for H.264 surveillance streams and is the most likely fix.

2.Use BT.709 color primaries/matrix for 1080p content (colour_primaries=1, transfer_characteristics=1, matrix_coefficients=1).

3.If they cannot change the encoder settings, they can try forcing software decoding in SmartClient as a workaround (this aligns with their observation that it works on setups without hardware acceleration).

The codebase parses these VUI parameters (CmH264VideoUsabilityInformation) and passes them through to both the Intel Media SDK decoder (MediaSDKVideoDecoder) and NVIDIA NVDEC decoder (NVDECVideoDecoder). The hardware decoder initialization paths don’t appear to have special handling for full-range YUV, which would explain why certain GPU drivers reject or mishandle the stream.