h.264 timestamps

MIP
,
1

I am trying to parse the h.264 stream from the GenericByteData content of the video stream packet using the protocol integration and libav to mux the frames in an mp4 container. I am able to split the packet along NALU lines. The problem is I don’t have the timestamps of the individual frames within the video block (GOP), just the timestamp of the first frame.

Is there a way to obtain the timestamps of the individual frames within the video block in order to generate the PTS (presentation time stamps)?

Thank you very much.

2

GBD has timestamp in each packet header, you should use that.

I wonder if there is more information here - https://developer.milestonesys.com/s/question/0D50O00003fox2oSAA/extracting-h264-from-genericbytedata

3

Thank you very much for your response. I already perused the post you provided the link for. I am able to get the timestamp from the header but it’s just one timestamp for the GOP.

I was wondering if I can get from the GBD for video blocks described in the documentation the start time, end time, and the number of pictures, and deduce from that the individual timestamps.

4

Do you think the approach outlined above, i.e. start time, end time, and number of frames, to deduce frame timestamps, work with variable frame rate?

Thank you very much for your help.

5

Hi Ruslan,

theoretically the start and end timestamps of a GOP combined with the frame count will give you the timestamp of the frame, however if you have the variable framerate that may affect the GOP size you may have wrong calculation. Especially if you have different framerates within the GOP. the GenericVideoStreamDataPacket contains a function GetTimeStampFrame which should give you the frames actual timestamp.

6

Hi Artur,

Can you please elaborate on how to use GenericVideoStreamDataPacket and where it’s defined. I cannot find any reference to it in the MIP SDK documentation.

Thank you very much.

7

GenericVideoStreamDataPacket is a internal c++ class not exposed in the MIP SDK. What we have in the MIP SDK is the documentation in “The GenericByteData format” and related pages. Perhaps you can extract the information needed.

8

Hello,

Thank you for your response. Am I correct in understanding that specifying in will result in video packets that contain a single frame/picture and the timestamp will represent the timestamp of that picture?

9

I expected to find and in the MIP documentation but failed in doing so. So for my understanding can you elaborate on your last question. If there is a reference to the documentation please let me know.

10

I apologize for not including a reference in my question. And it’s sorry for the misspelling. The Generic Byte Data Format documentation for Image Server - Requests and Responses, states:

  • If a client specified in its of the connect request, it will in the response receive multiple data packages similar to a multipart mime response.

My question is if the headers for those data packages will contain timestamp information.

Thank you very much.

11

Also can you please elaborate what are the connect options to get back from the image server a multi packet containing a single video packet and if that applies to h.264 streams. Would that packet contain a single frame?

From the documentation:

There is currently only one sub-format defined in addition to the three previously described sub-formats, namely the multi packet sub-format. A multi packet is a simple wrapper of other sub-formats. The multi packet can thus contain one or more single packets. The only two types of single packets that are now defined are the video stream packet sub-format and the audio stream packet sub-format. These two sub-formats are thus also single packet types that can be contained within a multi packet. The video block sub-format is not a single-packet type and thus can not be wrapped into a multi packet. Only one packet of each of the single packet type can be contained in the same multi packet. A multi packet must contain at least one packet. The multi packet sub-format is identified by the header data type being

0xFEF0

and is defined in more detail in the following.

Note:

Currently, multi packet wrapping is not used that much and should only occur for video streams where each video stream packet is send wrapped into a multi packet without any other single packets in the same multi packet and thus only containing one packet.