No NAL start found in demuxed packet

I've been using PyAV and aiortc to set up a video stream using webrtc. I want to repackage packets and send them without transcoding them. The issue I'm having is that aiortc is that the av_read_frame call doesn't generate a NAL start sequence, after which aiortc fails when trying to find.

I did a test, printing the first the beginning of each packer:

import av
container = av.open(file="jellyfish.mkv", format="matroska", mode="r")
video_stream = [x for x in container.streams if x.type == "video"]

for i in range(4):
    packet = next(container.demux(video_stream))
    s = bytes(packet)[0:8]
    print(s)

which yielded:

b'\x00\x00\xb5\xbae\x88\x80@'
b'\x00\x00A:A\x9a\x02\r'
b'\x00\x00\x18\xe2\x01\x9e\x04\x05'
b'\x00\x00\x19E\x01\x9e\x04\t' 

So it seems there is some sort of startcode, however not the one specified for NAL starts(0x000001 or 0x00000001): https://stackoverflow.com/a/23516925/3442097

Does anyone know what's at fault here?

MKV does not use annexb, while whatever packerger you are using does. You must convert the sizes to start codes.

Read this; Possible Locations for Sequence/Picture Parameter Set(s) for H.264 Stream