Can ffmpeg obtain the absolute timestamps of frames captured in a video stream?

2

I am a bit new to ffmpeg, apart from that I have a question. There are a lot of methods on superuser to obtain the timestamps of a frame in ffmpeg, however is it possible to get the absolute timestamp that is close to real time from a live video stream?

ffmpeg -i rtsp://admin:password@192.168.1.65:554 -vf "drawtext=fontfile=/usr/share/fonts/TTF/Vera.ttf: text='%{pts\:hms}': x=(w-tw)/2: y=h-(2*lh): fontcolor=white: box=1: boxcolor=0x00000000@1" image%03d.png

Such as in that command, PTS is used. I am wondering if we can print the PTS out to a text file and obtain the offsets in order to calculate the absolute time.

Edit: Sharing the report as a reply to Gyan

ffmpeg started on 2019-08-14 at 09:05:09
Report written to "ffmpeg-20190814-090509.log"
Command line:
ffmpeg -report -copyts -i rtsp://admin:password@192.168.1.65:554 -vf showinfo -an -vframes 10 -f null -
ffmpeg version 3.4.6-0ubuntu0.18.04.1 Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 7 (Ubuntu 7.3.0-16ubuntu3)
  configuration: --prefix=/usr --extra-version=0ubuntu0.18.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-librsvg --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable  WARNING: library configuration mismatch
  avcodec     configuration: --prefix=/usr --extra-version=0ubuntu0.18.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-librsvg --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61  libavutil      55. 78.100 / 55. 78.100
  libavcodec     57.107.100 / 57.107.100
  libavformat    57. 83.100 / 57. 83.100
  libavdevice    57. 10.100 / 57. 10.100
  libavfilter     6.107.100 /  6.107.100
  libavresample   3.  7.  0 /  3.  7.  0
  libswscale      4.  8.100 /  4.  8.100
  libswresample   2.  9.100 /  2.  9.100
  libpostproc    54.  7.100 / 54.  7.100
Splitting the commandline.
Reading option '-report' ... matched as option 'report' (generate a report) with argument '1'.
Reading option '-copyts' ... matched as option 'copyts' (copy timestamps) with argument '1'.
Reading option '-i' ... matched as input url with argument 'rtsp://admin:password@192.168.1.65:554'.
Reading option '-vf' ... matched as option 'vf' (set video filters) with argument 'showinfo'.
Reading option '-an' ... matched as option 'an' (disable audio) with argument '1'.
Reading option '-vframes' ... matched as option 'vframes' (set the number of video frames to output) with argument '10'.
Reading option '-f' ... matched as option 'f' (force format) with argument 'null'.
Reading option '-' ... matched as output url.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option report (generate a report) with argument 1.
Applying option copyts (copy timestamps) with argument 1.
Successfully parsed a group of options.
Parsing a group of options: input url rtsp://admin:password@192.168.1.65:554.
Successfully parsed a group of options.
Opening an input file: rtsp://admin:password@192.168.1.65:554.
[tcp @ 0x564a8770bac0] No default whitelist set
[rtsp @ 0x564a87709980] SDP:
v=0
o=- 1565773505153804 1565773505153804 IN IP4 192.168.1.65
s=Media Presentation
e=NONE
b=AS:5050
t=0 0
a=control:rtsp://192.168.1.65:554/
m=video 0 RTP/AVP 96
c=IN IP4 0.0.0.0
b=AS:5000
a=recvonly
a=x-dimensions:1920,1080
a=control:rtsp://192.168.1.65:554/trackID=1
a=rtpmap:96 H264/90000
a=fmtp:96 profile-level-id=420029; packetization-mode=1; sprop-parameter-sets=Z00AKZpkA8ARPy4C3AQEBQAAAwPoAADDUOhgAE+7AABPuku8uNDAAJ92AACfdJd5cKA=,aO48gA==
a=Media_header:MEDIAINFO=494D4B48010200000400000100000000000000000000000000000000000000000000000000000000;
a=appversion:1.0

[rtsp @ 0x564a87709980] video codec set to: h264
[rtsp @ 0x564a87709980] RTP Profile IDC: 42 Profile IOP: 0 Level: 29
[rtsp @ 0x564a87709980] RTP Packetization Mode: 1
[rtsp @ 0x564a87709980] Extradata set to 0x564a8770c3c0 (size: 62)
[rtp @ 0x564a8770db80] No default whitelist set
[udp @ 0x564a8770e3a0] No default whitelist set
[udp @ 0x564a8770e3a0] end receive buffer size reported is 131072
[udp @ 0x564a8771e8a0] No default whitelist set
[udp @ 0x564a8771e8a0] end receive buffer size reported is 131072
[rtsp @ 0x564a87709980] setting jitter buffer size to 500
[rtsp @ 0x564a87709980] hello state=0
[h264 @ 0x564a8770d400] nal_unit_type: 7, nal_ref_idc: 3
[h264 @ 0x564a8770d400] nal_unit_type: 8, nal_ref_idc: 3
[h264 @ 0x564a8770d400] nal_unit_type: 7, nal_ref_idc: 3
[h264 @ 0x564a8770d400] nal_unit_type: 8, nal_ref_idc: 3
[h264 @ 0x564a8770d400] nal_unit_type: 7, nal_ref_idc: 3
[h264 @ 0x564a8770d400] nal_unit_type: 8, nal_ref_idc: 3
[h264 @ 0x564a8770d400] nal_unit_type: 5, nal_ref_idc: 3
[h264 @ 0x564a8770d400] Reinit context to 1920x1088, pix_fmt: yuvj420p
[rtsp @ 0x564a87709980] max delay reached. need to consume packet
[rtsp @ 0x564a87709980] RTP: missed 34 packets
[h264 @ 0x564a8770d400] nal_unit_type: 7, nal_ref_idc: 3
[h264 @ 0x564a8770d400] nal_unit_type: 8, nal_ref_idc: 3
[h264 @ 0x564a8770d400] nal_unit_type: 5, nal_ref_idc: 3
[h264 @ 0x564a8770d400] bytestream overread -3
[h264 @ 0x564a8770d400] bytestream overread -11
[h264 @ 0x564a8770d400] error while decoding MB 56 44, bytestream -11
[h264 @ 0x564a8770d400] concealing 2873 DC, 2873 AC, 2873 MV errors in I frame
[h264 @ 0x564a8770d400] nal_unit_type: 1, nal_ref_idc: 3
[h264 @ 0x564a8770d400] Frame num gap 2 0
[h264 @ 0x564a8770d400] nal_unit_type: 1, nal_ref_idc: 3
[h264 @ 0x564a8770d400] nal_unit_type: 1, nal_ref_idc: 3
[h264 @ 0x564a8770d400] nal_unit_type: 1, nal_ref_idc: 3
[h264 @ 0x564a8770d400] nal_unit_type: 1, nal_ref_idc: 3
[rtsp @ 0x564a87709980] All info found
Input #0, rtsp, from 'rtsp://admin:password@192.168.1.65:554':
  Metadata:
    title           : Media Presentation
  Duration: N/A, start: 0.280000, bitrate: N/A
    Stream #0:0, 28, 1/90000: Video: h264 (Main), yuvj420p(pc, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
Successfully opened the file.
Parsing a group of options: output url -.
Applying option vf (set video filters) with argument showinfo.
Applying option an (disable audio) with argument 1.
Applying option vframes (set the number of video frames to output) with argument 10.
Applying option f (force format) with argument null.
Successfully parsed a group of options.
Opening an output file: -.
Successfully opened the file.
detected 4 logical cores
[h264 @ 0x564a8774c2e0] nal_unit_type: 7, nal_ref_idc: 3
[h264 @ 0x564a8774c2e0] nal_unit_type: 8, nal_ref_idc: 3
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> wrapped_avframe (native))
Press [q] to stop, [?] for help
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x564a8774c2e0] nal_unit_type: 7, nal_ref_idc: 3
[h264 @ 0x564a8774c2e0] nal_unit_type: 8, nal_ref_idc: 3
[h264 @ 0x564a8774c2e0] nal_unit_type: 5, nal_ref_idc: 3
[h264 @ 0x564a8774c2e0] Reinit context to 1920x1088, pix_fmt: yuvj420p
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x564a877cbea0] nal_unit_type: 7, nal_ref_idc: 3
[h264 @ 0x564a877cbea0] nal_unit_type: 8, nal_ref_idc: 3
[h264 @ 0x564a877cbea0] nal_unit_type: 5, nal_ref_idc: 3
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x564a8775f420] nal_unit_type: 1, nal_ref_idc: 3
[h264 @ 0x564a8775f420] Frame num gap 2 0
[h264 @ 0x564a877cbea0] bytestream overread -3
[h264 @ 0x564a877cbea0] bytestream overread -11
[h264 @ 0x564a877cbea0] error while decoding MB 56 44, bytestream -11
[h264 @ 0x564a877cbea0] concealing 2873 DC, 2873 AC, 2873 MV errors in I frame
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
[h264 @ 0x564a8786c220] nal_unit_type: 1, nal_ref_idc: 3
[h264 @ 0x564a877a7dc0] nal_unit_type: 1, nal_ref_idc: 3
[graph 0 input from stream 0:0 @ 0x564a877ab920] Setting 'video_size' to value '1920x1080'
[graph 0 input from stream 0:0 @ 0x564a877ab920] Setting 'pix_fmt' to value '12'
[graph 0 input from stream 0:0 @ 0x564a877ab920] Setting 'time_base' to value '1/90000'
[graph 0 input from stream 0:0 @ 0x564a877ab920] Setting 'pixel_aspect' to value '1/1'
[graph 0 input from stream 0:0 @ 0x564a877ab920] Setting 'sws_param' to value 'flags=2'
[graph 0 input from stream 0:0 @ 0x564a877ab920] Setting 'frame_rate' to value '25/1'
[graph 0 input from stream 0:0 @ 0x564a877ab920] w:1920 h:1080 pixfmt:yuvj420p tb:1/90000 fr:25/1 sar:1/1 sws_param:flags=2
[AVFilterGraph @ 0x564a8774fea0] query_formats: 3 queried, 2 merged, 0 already done, 0 delayed
[Parsed_showinfo_0 @ 0x564a87830660] config in time_base: 1/90000, frame_rate: 25/1
[Parsed_showinfo_0 @ 0x564a87830660] config out time_base: 0/0, frame_rate: 0/0
[Parsed_showinfo_0 @ 0x564a87830660] n:   0 pts:  25200 pts_time:0.28    pos:       -1 fmt:yuvj420p sar:1/1 s:1920x1080 i:P iskey:1 type:I checksum:9367EFC4 plane_checksum:[54B3B719 FEC9788F EF1BC00D] mean:[98 130 128 ] stdev:[72.2 13.2 9.4 ]
Output #0, null, to 'pipe:':
  Metadata:
    title           : Media Presentation
    encoder         : Lavf57.83.100
    Stream #0:0, 0, 1/25: Video: wrapped_avframe, yuvj420p, 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc
    Metadata:
      encoder         : Lavc57.107.100 wrapped_avframe
[h264 @ 0x564a8774c2e0] nal_unit_type: 1, nal_ref_idc: 3
[Parsed_showinfo_0 @ 0x564a87830660] n:   1 pts:   3600 pts_time:0.04    pos:       -1 fmt:yuvj420p sar:1/1 s:1920x1080 i:P iskey:1 type:I checksum:0459B9DA plane_checksum:[541A6FB4 BBA591AB 8FF6B86C] mean:[98 130 128 ] stdev:[72.2 13.2 9.4 ]
[null @ 0x564a87748360] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 7 >= 1
[h264 @ 0x564a877cbea0] nal_unit_type: 1, nal_ref_idc: 3
[Parsed_showinfo_0 @ 0x564a87830660] n:   2 pts:  10800 pts_time:0.12    pos:       -1 fmt:yuvj420p sar:1/1 s:1920x1080 i:P iskey:0 type:P checksum:9064E408 plane_checksum:[78458BB3 40DBC2A9 6B31959D] mean:[98 130 128 ] stdev:[72.3 13.2 9.4 ]
[null @ 0x564a87748360] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 7 >= 3
[h264 @ 0x564a8775f420] nal_unit_type: 1, nal_ref_idc: 3
[Parsed_showinfo_0 @ 0x564a87830660] n:   3 pts:  14400 pts_time:0.16    pos:       -1 fmt:yuvj420p sar:1/1 s:1920x1080 i:P iskey:0 type:P checksum:3D11A586 plane_checksum:[F2931EDC A19B16ED 3E396FBD] mean:[98 130 128 ] stdev:[72.3 13.1 9.4 ]
[null @ 0x564a87748360] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 7 >= 4
[h264 @ 0x564a8786c220] nal_unit_type: 1, nal_ref_idc: 3
[Parsed_showinfo_0 @ 0x564a87830660] n:   4 pts:  18000 pts_time:0.2     pos:       -1 fmt:yuvj420p sar:1/1 s:1920x1080 i:P iskey:0 type:P checksum:8F4BB787 plane_checksum:[8B24254D 7A06967C 2B01FBAF] mean:[98 130 128 ] stdev:[72.3 13.1 9.3 ]
[null @ 0x564a87748360] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 7 >= 5
[h264 @ 0x564a877a7dc0] nal_unit_type: 1, nal_ref_idc: 3
[Parsed_showinfo_0 @ 0x564a87830660] n:   5 pts:  21600 pts_time:0.24    pos:       -1 fmt:yuvj420p sar:1/1 s:1920x1080 i:P iskey:0 type:P checksum:C29B5361 plane_checksum:[CA42C1C5 5926F77B AE559A03] mean:[97 130 128 ] stdev:[72.4 13.0 9.2 ]
[null @ 0x564a87748360] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 7 >= 6
[h264 @ 0x564a8774c2e0] nal_unit_type: 1, nal_ref_idc: 3
[Parsed_showinfo_0 @ 0x564a87830660] n:   6 pts:  25200 pts_time:0.28    pos:       -1 fmt:yuvj420p sar:1/1 s:1920x1080 i:P iskey:0 type:P checksum:CB39A519 plane_checksum:[0807032F 9E074697 7AED5B53] mean:[97 130 128 ] stdev:[72.4 13.0 9.2 ]
[null @ 0x564a87748360] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 7 >= 7
[h264 @ 0x564a877cbea0] nal_unit_type: 1, nal_ref_idc: 3
[Parsed_showinfo_0 @ 0x564a87830660] n:   7 pts:  28800 pts_time:0.32    pos:       -1 fmt:yuvj420p sar:1/1 s:1920x1080 i:P iskey:0 type:P checksum:50254073 plane_checksum:[8B3EC5DF B0665A0E 324E2077] mean:[97 130 128 ] stdev:[72.2 13.0 9.2 ]
[h264 @ 0x564a8775f420] nal_unit_type: 1, nal_ref_idc: 3
[Parsed_showinfo_0 @ 0x564a87830660] n:   8 pts:  32400 pts_time:0.36    pos:       -1 fmt:yuvj420p sar:1/1 s:1920x1080 i:P iskey:0 type:P checksum:71AD2EED plane_checksum:[D71FCA9D 3E8A10F4 C5DB534D] mean:[96 130 128 ] stdev:[71.9 13.0 9.2 ]
[h264 @ 0x564a8786c220] nal_unit_type: 1, nal_ref_idc: 3
[Parsed_showinfo_0 @ 0x564a87830660] n:   9 pts:  36000 pts_time:0.4     pos:       -1 fmt:yuvj420p sar:1/1 s:1920x1080 i:P iskey:0 type:P checksum:DF46AA3F plane_checksum:[96F45F0F 110D9201 2D96B920] mean:[96 130 128 ] stdev:[71.6 13.0 9.2 ]
No more output streams to write to, finishing.
[Parsed_showinfo_0 @ 0x564a87830660] n:  10 pts:  39600 pts_time:0.44    pos:       -1 fmt:yuvj420p sar:1/1 s:1920x1080 i:P iskey:0 type:P checksum:B135E91E plane_checksum:[BA829D19 1C3A3ED3 84F10D32] mean:[95 130 128 ] stdev:[71.2 13.0 9.2 ]
frame=   10 fps=0.0 q=-0.0 Lsize=N/A time=00:00:00.44 bitrate=N/A speed=3.86x    
video:5kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Input file #0 (rtsp://admin:password@192.168.1.65:554):
  Input stream #0:0 (video): 14 packets read (404551 bytes); 11 frames decoded; 
  Total: 14 packets (404551 bytes) demuxed
Output file #0 (pipe:):
  Output stream #0:0 (video): 10 frames encoded; 10 packets muxed (5280 bytes); 
  Total: 10 packets (5280 bytes) muxed
11 frames successfully decoded, 0 decoding errors
video
ffmpeg
streaming
asked on Super User Aug 13, 2019 by penguin • edited Aug 14, 2019 by penguin

1 Answer

1

You can extract absolute timestamp from RTCP Sender Report, but this functionality is not available in library API, you have to use header libavformat/rtsp.h in order to access internal data structures. And then you have to calculate the ntp timestamp for every frame:

RTSPState* rtsp_state = (RTSPState*) pFormatCtx->priv_data;
RTSPStream* rtsp_stream = rtsp_state->rtsp_streams[0];
RTPDemuxContext* rtp_demux_context = (RTPDemuxContext*) rtsp_stream->transport_priv;

int32_t d_ts = rtp_demux_context->timestamp - rtp_demux_context->last_rtcp_timestamp;
uint64_t last_ntp_time = rtp_demux_context->last_rtcp_ntp_time;
uint32_t seconds = ((last_ntp_time >> 32) & 0xffffffff)-2208988800;
uint32_t fraction  = (last_ntp_time & 0xffffffff);
double useconds = ((double) fraction / 0xffffffff);
double base_time = seconds+useconds;

double frame_ntp_time = base_time+d_ts/90000.0;

You can find full example here.

answered on Super User May 24, 2020 by Islam Sabyrgaliyev • edited Jun 12, 2020 by Community

User contributions licensed under CC BY-SA 3.0