MP4 / MOV H.264 - determine bytestream standard (Annex B or AVC)

how can I figure out if the video bytestream of a MP4 or MOV H.264 videofile is decoded using the AVC standard or the Annex B standard? Both standards can start their NALUs with 0x000001.

In the Annex B Standard every NALU starts with a start code 0x000001 or 0x00000001. In the AVC standard the length field can also be 0x00000134. Is there any additional information located in the MP4 file, which give me the hint, that the start bytes (0x00000134) are the length field of the AVC standard and not the start code 0x000001 of a Annex B standard video?

In my specific example, my MP4 video file has the AVC standard and my MOV file has the Annex B Standard. Both have the "NALULengthSizeMinusOne" field set to the value 4.

"Annex B" looks usually like this:

(AUD)(SPS)(PPS)(I-Slice)(PPS)(P-Slice)(PPS)(P-Slice) ... (AUD)(SPS)(PPS)(I-Slice)

"AVC" or "MP4" has usually the (AUD)(SPS)(PPS) removed. Annex B should start 00 00 00 01 09 00 00 00 01 x8.

So you could look for the start code prefix AUDs, SPSs and PPSs.

For "AVC" you can read the size field than verify that the next byte consists of a valid NAL unit type and than seek past the NAL and do the same a few time.