Format h264 detected only with low score of 1, misdetection possible

there are a h264 rtp stream passing through server, I catch every rtp packets and extract every nalu by getting rid of rtp header, and then write every nalu to file record.h264 with adding a h264 prefix start code 0x00000001, but file record.h264 cannot be played by ffplay and cannot be parsed by ffprobe, where is wrong?

here is my record file:https://github.com/sshsu/record_h264_file

You might be skipping the decoding of the NAL header within the RtpPacket to it's full format from the aggregated form.

I have an implementation in C# here: https://github.com/juliusfriedman/net7mma_core/blob/master/RtspServer/MediaTypes/RFC6184Media.cs if it helps you.