ffmpeg creates an mp4 stream which results error in Firefox

0

I'm about to play an fmp4 in HTML5 video element.

I was successfully created a websocket to pass ffmpeg's output into MSE.

However when I try to open the page in Firefox (72.0.1, 64bit, under Ubuntu 18.04LTS), it always results an error:

Media resource blob:http://localhost/XXXX could not be decoded, error: Error Code: NS_ERROR_FAILURE (0x80004005)
Details: virtual mozilla::MediaResult mozilla::MP4ContainerParser::IsInitSegmentPresent(const mozilla::MediaSpan &): Invalid Top-Level Box:f"

This is my FFMPEG's line:

ffmpeg -r 5 -i rtsp://IPCAMERA -c:v copy -an -movflags +frag_keyframe+empty_moov+default_base_moof -f mp4 pipe:1

This is how the server side Java (with Tomcat engine) parses the output of this command (this might be inefficient but for now is ok):

ProcessBuilder b = new ProcessBuilder(FFMPEGCOMMAND.split("\\s+"));
try {
    p = b.start();
} catch (IOException e) {
    e.printStackTrace();
}
InputStream input = p.getInputStream();
int bytes_read = 0;
byte buffer[] = new byte[512];
try {
    while (0 < (bytes_read = input.read(buffer, 0, 512))) {
        System.out.println("Bytes read:" + bytes_read);
        this.session.getBasicRemote().sendBinary(ByteBuffer.wrap(buffer));
    }
} catch (IOException e) {
    e.printStackTrace();
}

Then the client side is a websocket-MSE like in this repo.

Results:

  1. When I debug the server side, and there is a breakpoint before the sendBinary call, and I wait for some seconds before letting run the server side, then a first picture is shown in the browser, but then immediately goes onto the error above.

  2. If I run the server side (without any breakpoints), the browser does not show any picture, it immediately goes onto the error.

Invalid Top-Level box error message is always followed by (a) random garbage character(s).

The proof that this should be working is in the point 1. If I wait some time before letting roll the data to the client, it can decode 1 (or maybe more frames) before reaching that error.

This might be an error with my ffmpeg command-line.

However I could not really find any good resources on this topic (only found those ones which related to older releases of Firefox).

Update1

Here is the FFMPEG log when the same command creates an mp4 file instead of the pipe: https://pastebin.com/Gjq2vxeT

Here is the detailed Firefox log with the wrong box:

Details: virtual mozilla::MediaResult mozilla::MP4ContainerParser::IsInitSegmentPresent(const mozilla::MediaSpan &): Invalid Top-Level Box:f

Please note, Top-Level Box is always 'f' when I'm doing the scenario marked in point 2 (running without breakpoints).

Update2:

Here are the ffmpeg's current output (with numeric and alphanumeric representation, for the first 128 items): https://pastebin.com/DeJMfNYs

The interesting thing is that the first 4 bytes seems invalid for me. However starting from byte4 (5th byte), it seems OK ("ftyp").

Could you please confirm this?

ffmpeg
mp4
fmp4
asked on Stack Overflow Jan 22, 2020 by Daniel • edited Jan 23, 2020 by Daniel

1 Answer

1

The problem was on the server's side.

With this line:

this.session.getBasicRemote().sendBinary(ByteBuffer.wrap(buffer));

This is not good because when there is a 0 in the buffer, ByteBuffer's wrap method stops.

The good one is this:

this.session.getBasicRemote().sendBinary(ByteBuffer.wrap(buffer, 0, bytes_read));

What a silly mistake - now works fine :)

answered on Stack Overflow Jan 23, 2020 by Daniel

User contributions licensed under CC BY-SA 3.0