How to get real-time video frame data in YCbCr420_Planar format?

Here’s my code to fetch the frame data from BitmapLiveSouce.

private void OpenLiveSession()
{
    if (_viewItemManager.Config.DeviceId != null)
    {
        _selectedItem = Configuration.Instance.GetItem(_viewItemManager.Config.DeviceId, Kind.Camera);
        if (_selectedItem != null)
        {
            _bitmapLiveSource = new BitmapLiveSource(_selectedItem, BitmapFormat.YCbCr420_Planar);
            _bitmapLiveSource.LiveContentEvent += BitmapLiveSourceLiveContentEvent;
            _bitmapLiveSource.Width = (int)imageBox.Width;
            _bitmapLiveSource.Height = (int)imageBox.Height;
            _bitmapLiveSource.SetKeepAspectRatio(true, true);
            _bitmapLiveSource.Init();
            _bitmapLiveSource.LiveModeStart = true;
            _bitmapLiveSource.SingleFrameQueue = true;     
        }
    }
}
 
void BitmapLiveSourceLiveContentEvent(object sender, EventArgs e)
{
    try
    {
        if (!Dispatcher.CheckAccess())
        {
            // Make sure we execute on the UI thread before updating UI Controls
            Dispatcher.Invoke(new EventHandler(BitmapLiveSourceLiveContentEvent), new[] { sender, e });
        }
        else
        {
            var args = e as LiveContentEventArgs;
            if (args != null)
            {
                if (args.LiveContent != null)
                {
                    var bitmapContent = args.LiveContent as LiveSourceBitmapContent;
                    if (bitmapContent != null)
                    {
                        if (_stopLive || imageBox.Width == 0 || imageBox.Height == 0)
                        {
                            bitmapContent.Dispose();
                        }
                        else
                        {
                            int width = bitmapContent.GetPlaneWidth(0);
                            int height = bitmapContent.GetPlaneHeight(0);
                            int stride = bitmapContent.GetPlaneStride(0);
                            byte[] dateY = bitmapContent.GetPlaneBytes(0);
                            byte[] dateU = bitmapContent.GetPlaneBytes(1);
                            byte[] dateV = bitmapContent.GetPlaneBytes(2);
                            //IntPtr dataY = bitmapContent.GetPlanePointer(0);
                            //IntPtr dataU = bitmapContent.GetPlanePointer(1);
                            //IntPtr dataV = bitmapContent.GetPlanePointer(2);
							
                            int ySize = stride * height;
                            int uvSize = (width / 2) * (height / 2);
 
                            string filePath = "./test.yuv";
                            using (FileStream fileStream = new FileStream(filePath, FileMode.Append, FileAccess.Write))
                            {
                                fileStream.Write(dateY, 0, ySize);
                                fileStream.Write(dateU, 0, uvSize);
                                fileStream.Write(dateV, 0, uvSize);
                            }
 
                            bitmapContent.Dispose();
                        }
                    }
                }
            }
        }
    }
    catch (Exception ex)
    {
        EnvironmentManager.Instance.ExceptionDialog("BitmapLiveSourceLiveContentEvent", ex);
    }
}

The data retrieved in the above way is saved to a file, and there are two scenarios:

1. When SmartClient is in full-screen mode and only one video window is open, the obtained frame data is normal.

2. When SmartClient is not full-screen or multiple video windows are opened, the y-component of the obtained frame data is normal, and the uv-component is abnormal.

How do I get the correct YCbCr420_Planar format frame data.

A deeper investigation might be needed but allow me some questions first.

imageBox is used to set width and height, does this one change as the Smart Client (SC) is resized, maximized or multiple video windows are opened? Can you make a run where you log those values and the data values that are wrong (and perhaps include expected values)? We would like to see data on a correct and an incorrect.

What version is the SC?

The camera you are testing with is it setup to have more than one live stream? (In Management Client pick the camera and inspect Streams tab in the properties window.)

I only had one video window open;

I use XProtect Smart Client 2024 R1;

I was able to solve the above problem by adjusting the width to a multiple of 64.

Now there’s a new question:

Take the frame data according to the way I did above, sometimes there will be frame data with a blurred screen, how to solve this problem?This is also the case with RGBVIdeoEnhancement.

The graphics card used by the computer is Intel(R) UHD Graphics 750.

Here’s the data I’ve saved, at 1600x816, and it’s abnormal at frames 21-24.

Even if I get an expert to analyze the data and we see the abnormality I don’t think it will tell us anything about how the data got bad.

Perhaps you can describe how you reproduce this issue using the RGBVIdeoEnhancement, if I then can reproduce the issue in the Milestone test lab, there is a chance we can figure out what the problem is, and perhaps also how to solve it.

When you describe it, please also answer these questions.

You say sometimes, is there a pattern to it or does it simply happen at random times?

When you see the abnormality, blurred frame of video, it could be originating from the source of video, the camera. It would be helpful if we can rule out this source of error.

Do you ever notice any issue watching the camera stream using the regular camera views in the Smart Client?

If you have two test cameras, do you see the issue on both of them?

If you change the settings (resolution, bitrate or something else) on a test camera can you make the issue go away?

I open a video window, use the RGBVIdeoEnhancement example plug-in, select a camera and do nothing, after a while, there will be a blur screen, once the first time there is a blur screen, then it will appear frequently.

The blur screen does not come from the camera, and it is normal to use the regular camera view to watch the camera stream in the smart client;

The resolution of the camera with the blurred screen in my test is 4000*3000;

When I changed the resolution to 1280*960, there was no blurred screen phenomenon for the time being.

I don’t want to have to change the configuration every time I use the plugin, is there any other way to fix the above problem?

Can you please comment on Width and Height while changing the SC size, this is no longer an issue in regards to the “blurry”?

Please make a screen capture or similar to allow us to see the blurry, it might be a help for us to see.

SC size(1600x816)​

Normal image

Abnormal image

Can you please comment on Width and Height while changing the SC size, this is no longer an issue in regards to the “blurry”?