C++ Media ToolKit - Raw Image

MIP
, , ,
1

Hi,

I am developing C++ application to get live streaming of cameras. I am trying to modify existing MediaLiveServiceJPEG example to get raw images.

I want data in unsigned char* BGR24 format so that I can use it with OpenCV

Here is how am trying to modify config.

utf8_string_t config = "<?xml version='1.0' encoding='utf-8'?>"
                         "<toolkit type='source'>"
                         "  <provider>mmp</provider>"
                         "  <config>"
						 "    <image_color_format_convert format = 'BGR24'> "
                         //"    <jpeg_encoder quality='90' quality_update_key='qual'>"
                         "      <video_decoder number_of_threads='-1'>"
                         "        <toolkit type='source'>"
                         "          <provider>is</provider>"
                         "          <config>"
						 "            <server_uri>"+ vmsRecorderUri +"</server_uri>"
                         "            <device_id>"+ cameraGuid +"</device_id>"
                         "            <media_type>VIDEO</media_type>" + authorizationToken +
                         "            <maximum_queue_size>5</maximum_queue_size>"
                         "          </config>"
                         "        </toolkit>"
                         "      </video_decoder>"
                         //"    </jpeg_encoder>"
			             "    </image_color_format_convert> "
                         "  </config>"
                         "</toolkit>";

Even though, frame is received, am not able to actually read data. May I also please know how to get image width, height and fps?

I will really appreciate any kind of help.

@Bo Andersen​ ,@Michael Wahlstrøm (Milestone Systems)​ , will you please help ?

2

About Media Toolkit (a tutorial)-

http://doc.developer.milestonesys.com/html/index.html?base=mmtkhelp/tutorial.html#connecting_to_a_source&tree=tree_2.html

Note that what I understand from what you are writing you do not want raw data but the images in bitmap format. This should be achievable.

3

Hi @Bo Ellegård Andersen (Milestone Systems)​ , Thanks for the reply. I have gone through the shared link. Yes I want bitmap image data in BGR24 format instead of raw data. I feel, I need to use image_color_format_convert transformation from Media Processing Source Toolkit.

I have modified existing MediaLiveServiceJPEG.cpp file as follows.

// Downcast to live source toolkit interface
ImLiveSourceToolkit *liveSourceToolkit = ynamic_cast<ImLiveSourceToolkit *>(sourceToolkit);
        if (liveSourceToolkit == 0)
          cerr << "Created source toolkit does not implement the live source toolkit interface!" << endl;
        else
        {
		  std::wcerr << "... Starting video stream" << std::endl;
                  liveSourceToolkit->StartLiveStream();
		  long ix = 1;
		  DWORD ftyp = GetFileAttributesA("C:\\Data");
		  if (ftyp == INVALID_FILE_ATTRIBUTES) {
			  CreateDirectory(L"C:\\Data", NULL);
		  }
		  unsigned char* imageData;
		  int count = 0;
          while (!_kbhit())
          {
                   ImLiveSourceToolkit::get_live_status_t liveDataResult = 
                   liveSourceToolkit->GetLiveData(data, 10000);
            if (liveDataResult == ImLiveSourceToolkit::LIVE_DATA_RETRIEVED)
            {
                    mImMediaData *mediaData = dynamic_cast<ImMediaData *>(data);
	            mediaData->GetProperty()
                    if (mediaData != 0)
                    {
				std::wcerr << "... Frame Received" << std::endl;
                		const byte* b1 = mediaData->GetMediaBodyPointer();
				size_t b2 = mediaData->GetMediaBodySize();
				if (count == 0) {
					imageData = (unsigned char*)malloc(sizeof(unsigned char)*576*960*3);
					count = 1;
				}
				memcpy(imageData, mediaData->GetMediaBodyPointer(), mediaData->GetMediaBodySize());
				cv::Mat Img(576, 960, CV_8UC3,data);
				imshow("image", Img);
				waitKey(0);

I am able to receive frame, However, mediaData->GetMediaBodyPointer() returns null and size from mediaData->GetMediaBodySize() is also zero.

May I please know what I am doing wrong in the code. also, I am still clueless how to get width and height of received image.

4

Hi Ankita,

I think your issue is that you need to cast it to ImImage class as this is the actual data that you get.

Please read more here:

http://doc.developer.milestonesys.com/html/index.html?base=mmtkhelp/tutorial.html#connecting_to_a_source&tree=tree_2.html

5

Hi @Artur Magaljan (Milestone Systems)​ ,

Yes. The problem was indeed ImImage. I am now getting output as expected.

I have one small question though. Does Milestone add padding to images? ImImage has two functions getimagewidth() and getplanestride(). For few videos I checked, there were no extra bits.

6

Hi @[Artur Magaljan]​ ,

Yes. The problem was indeed ImImage. I am now getting output as expected. Thanks a lot :slight_smile:

I have one small question though. Does Milestone add padding to images? ImImage has two functions getimagewidth() and getplanestride(). For few videos I checked, there were no extra bits.

7

Hi Ankita,

It depends on the format and it it is needed. we do not add anything to images if not a “must”.

But i guess for your needs you can just call getimagewidth and getimageheight to get the resolution of the image.

Artur.