Why are no dts pts written to my mp4 container

Based on my (self-answered) question here Muxing AVPackets into mp4 file - revisited, I have to ask, what could be the reason why there are no values written for pts/dts in the resulting mp4 container.

I examined the container file with the tool MediaInfo. I observe that only the very first Frame contains a value for pts in the container. After that, pts is not even shown in the mp4 file anymore, but dts is, with a value of all zeros.

This is the output from MediaInfo for the first 3 frames:

0000A2   slice_layer_without_partitioning (IDR) - 0 (0x0) - Frame 0 - slice_type I - frame_num 0 - DTS 00:00:00.000 - PTS 00:00:00.017 (141867 bytes)
0000A2    Header (5 bytes)
0000A2     zero_byte:                          0 (0x00)
0000A3     start_code_prefix_one_3bytes:       1 (0x000001)
0000A6     nal_ref_idc:                        3 (0x3) - (2 bits)
0000A6     nal_unit_type:                      5 (0x05) - (5 bits)
0000A7    slice_header (3 bytes)
0000A7     first_mb_in_slice:                  0 (0x0)
0000A7     slice_type:                         7 (0x07) - I
0000A8     pic_parameter_set_id:               0 (0x0)
0000A8     frame_num:                          0 (0x0)
0000A8     idr_pic_id:                         0 (0x0)
0000A8     no_output_of_prior_pics_flag:       No
0000A8     long_term_reference_flag:           No
0000A9     slice_qp_delta:                     -5 (0xFFFFFFFB)
0000AA     disable_deblocking_filter_idc:      0 (0x0)
0000AA     slice_alpha_c0_offset_div2:         0 (0x0)
0000AA     slice_beta_offset_div2:             0 (0x0)
0000AA    slice_data (141856 bytes)
0000AA     (ToDo):                             (Data)
022ACD   slice_layer_without_partitioning (IDR) - 0 (0x0) - Frame 0 - slice_type I - frame_num 0 - DTS 00:00:00.000 - PTS 00:00:00.017 - first_mb_in_slice 8040 (2248 bytes)
022ACD    Header (5 bytes)
022ACD     zero_byte:                          0 (0x00)
022ACE     start_code_prefix_one_3bytes:       1 (0x000001)
022AD1     nal_ref_idc:                        3 (0x3) - (2 bits)
022AD1     nal_unit_type:                      5 (0x05) - (5 bits)
022AD2    slice_header (6 bytes)
022AD2     first_mb_in_slice:                  8040 (0x001F68)
022AD5     slice_type:                         7 (0x07) - I
022AD6     pic_parameter_set_id:               0 (0x0)
022AD6     frame_num:                          0 (0x0)
022AD6     idr_pic_id:                         0 (0x0)
022AD6     no_output_of_prior_pics_flag:       No
022AD6     long_term_reference_flag:           No
022AD7     slice_qp_delta:                     -5 (0xFFFFFFFB)
022AD8     disable_deblocking_filter_idc:      0 (0x0)
022AD8     slice_alpha_c0_offset_div2:         0 (0x0)
022AD8     slice_beta_offset_div2:             0 (0x0)
022AD8    slice_data (2237 bytes)
022AD8     (ToDo):                             (Data)
023395  1 (36212 bytes)
023395   slice_layer_without_partitioning (non-IDR) - 2 (0x2) - Frame 1 - slice_type P - frame_num 1 - DTS 00:00:00.000 (36017 bytes)
023395    Header (5 bytes)
023395     zero_byte:                          0 (0x00)
023396     start_code_prefix_one_3bytes:       1 (0x000001)
023399     nal_ref_idc:                        3 (0x3) - (2 bits)
023399     nal_unit_type:                      1 (0x01) - (5 bits)
02339A    slice_header (3 bytes)
02339A     first_mb_in_slice:                  0 (0x0)
02339A     slice_type:                         5 (0x5) - P
02339A     pic_parameter_set_id:               0 (0x0)
02339A     frame_num:                          1 (0x1)
02339B     num_ref_idx_active_override_flag (0 bytes)
02339B      num_ref_idx_active_override_flag:  Yes
02339B      num_ref_idx_l0_active_minus1:      0 (0x0)
02339B     ref_pic_list_modification_flag_l0:  No
02339B     adaptive_ref_pic_marking_mode_flag: No
02339C     cabac_init_idc:                     0 (0x0)
02339C     slice_qp_delta:                     -3 (0xFFFFFFFD)
02339C     disable_deblocking_filter_idc:      0 (0x0)
02339C     slice_alpha_c0_offset_div2:         0 (0x0)
02339D     slice_beta_offset_div2:             0 (0x0)
02339D    slice_data (36012 bytes)
02339D     (ToDo):                             (Data)
02C046   slice_layer_without_partitioning (non-IDR) - 2 (0x2) - Frame 1 - slice_type P - frame_num 1 - DTS 00:00:00.000 - first_mb_in_slice 8040 (195 bytes)
02C046    Header (5 bytes)
02C046     zero_byte:                          0 (0x00)
02C047     start_code_prefix_one_3bytes:       1 (0x000001)
02C04A     nal_ref_idc:                        3 (0x3) - (2 bits)
02C04A     nal_unit_type:                      1 (0x01) - (5 bits)
02C04B    slice_header (6 bytes)
02C04B     first_mb_in_slice:                  8040 (0x001F68)
02C04E     slice_type:                         5 (0x5) - P
02C04E     pic_parameter_set_id:               0 (0x0)
02C04E     frame_num:                          1 (0x1)
02C04F     num_ref_idx_active_override_flag (0 bytes)
02C04F      num_ref_idx_active_override_flag:  Yes
02C04F      num_ref_idx_l0_active_minus1:      0 (0x0)
02C04F     ref_pic_list_modification_flag_l0:  No
02C04F     adaptive_ref_pic_marking_mode_flag: No
02C050     cabac_init_idc:                     0 (0x0)
02C050     slice_qp_delta:                     -3 (0xFFFFFFFD)
02C050     disable_deblocking_filter_idc:      0 (0x0)
02C050     slice_alpha_c0_offset_div2:         0 (0x0)
02C051     slice_beta_offset_div2:             0 (0x0)
02C051    slice_data (190 bytes)
02C051     (ToDo):                             (Data)
02C109  1 (26280 bytes)
02C109   slice_layer_without_partitioning (non-IDR) - 4 (0x4) - Frame 2 - slice_type P - frame_num 2 - DTS 00:00:00.000 (26157 bytes)
02C109    Header (5 bytes)
02C109     zero_byte:                          0 (0x00)
02C10A     start_code_prefix_one_3bytes:       1 (0x000001)
02C10D     nal_ref_idc:                        3 (0x3) - (2 bits)
02C10D     nal_unit_type:                      1 (0x01) - (5 bits)
02C10E    slice_header (3 bytes)
02C10E     first_mb_in_slice:                  0 (0x0)
02C10E     slice_type:                         5 (0x5) - P
02C10E     pic_parameter_set_id:               0 (0x0)
02C10E     frame_num:                          2 (0x2)
02C10F     num_ref_idx_active_override_flag (0 bytes)
02C10F      num_ref_idx_active_override_flag:  Yes
02C10F      num_ref_idx_l0_active_minus1:      0 (0x0)
02C10F     ref_pic_list_modification_flag_l0:  No
02C10F     adaptive_ref_pic_marking_mode_flag: No
02C110     cabac_init_idc:                     0 (0x0)
02C110     slice_qp_delta:                     -2 (0xFFFFFFFE)
02C110     disable_deblocking_filter_idc:      0 (0x0)
02C110     slice_alpha_c0_offset_div2:         0 (0x0)
02C111     slice_beta_offset_div2:             0 (0x0)
02C111    slice_data (26152 bytes)
02C111     (ToDo):                             (Data)
032736   slice_layer_without_partitioning (non-IDR) - 4 (0x4) - Frame 2 - slice_type P - frame_num 2 - DTS 00:00:00.000 - first_mb_in_slice 8040 (123 bytes)
032736    Header (5 bytes)
032736     zero_byte:                          0 (0x00)
032737     start_code_prefix_one_3bytes:       1 (0x000001)
03273A     nal_ref_idc:                        3 (0x3) - (2 bits)
03273A     nal_unit_type:                      1 (0x01) - (5 bits)
03273B    slice_header (6 bytes)
03273B     first_mb_in_slice:                  8040 (0x001F68)
03273E     slice_type:                         5 (0x5) - P
03273E     pic_parameter_set_id:               0 (0x0)
03273E     frame_num:                          2 (0x2)
03273F     num_ref_idx_active_override_flag (0 bytes)
03273F      num_ref_idx_active_override_flag:  Yes
03273F      num_ref_idx_l0_active_minus1:      0 (0x0)
03273F     ref_pic_list_modification_flag_l0:  No
03273F     adaptive_ref_pic_marking_mode_flag: No
032740     cabac_init_idc:                     0 (0x0)
032740     slice_qp_delta:                     -2 (0xFFFFFFFE)
032740     disable_deblocking_filter_idc:      0 (0x0)
032740     slice_alpha_c0_offset_div2:         0 (0x0)
032741     slice_beta_offset_div2:             0 (0x0)
032741    slice_data (118 bytes)
032741     (ToDo):                             (Data)
0327B1  1 (21125 bytes)

It goes on like that, even though I set pts and dts. The settings may not be correct already (I do some calculations like (1 / framerate) * FrameNumber), but I would expect at least some numbers in pts and dts, when I set the according fields in the avPacket structure and write that via av_interleaved_write_frame(outFmtCtx, &avPacket); to the file.

What could be wrong here?

Edit:

(please see below in the comments the download to my testdata and source file) One thing that bugs me is the fact, if I compare the output of MediaInfo from my file and that of the muxing.c generated is, that in the header, the muxing.c generated already mentions the duration of the file as 9960 ms, whereas mine is only 40 ms.

muxing.c also does call avformat_write_header before even one frame is drawn. Yes, I suppose that the header will be updated, when the either av_interleaved_write_frame or av_write_trailer is called, but I totally not understand the mechanics behind it. Maybe somebody can enlighten me with some background information of any kind.

Additionally, I think it could be necessarry to extract some SPS and PPS from my raw data (preceding the I-slice), and give that as extra data to the avformat_write_header call. But I just cannot figure out myself if I have to do that at all and if so, how to do it.