C# marshal struct with complex array to bytes

I have to interop with a device, through a C++ interface that accepts byte[] for both in/out streams.

Many of the data being passed around are in the form of an outer struct containing some numbers, and a dynamic array of inner structs (which in turn may contain arrays of their own). A simple form would be:

[StructLayout(LayoutKind.Sequential, Pack = 1)]
public struct Outer
{
    public float Outer_1;
    //...
    public double Outer_N;

    public Inner[] InnerArray; // variable length
}

[StructLayout(LayoutKind.Sequential, Pack = 1)]
public struct Inner
{
    public Int16 Inner_1;
    //...
    public Int32 Inner_N;
}

Simple nested (non-arrayed) structs work fine, either using a fixed(byte*) + Marshal.StructureToPtr(), or GCHandle.Alloc() + Marshal.Copy().

I do know that the array is just a pointer, and its contents would need separate marshalling; but I was hoping I could at least start by marshalling the non-array Outer_1 .. Outer_N part of the structure, followed by a loop to marshal the array contents to different locations of the byte array. I'm trying to avoid having to marshal Outer_1..N one-by-one.

And this is where things fall apart. Neither of the usual methods work.

[Fact]
public unsafe void TestSerialize()
{
    Outer managed = new()
    {
        Outer_1 = 1.0f,
        Outer_N = 3.14,
        InnerArray = new[] {
            new Inner { Inner_1 = 1, Inner_N = Int32.MinValue },
            new Inner { Inner_1 = 10, Inner_N = Int32.MaxValue },
        },
    };

    var buffer = new byte[500]; // Could be calculated, but won't bother right now.

    //--- Attempt 1:
    fixed (byte* pBuf = buffer)
    {
        //System.NotSupportedException: 'Operation is not supported. (0x80131515)'
        Marshal.StructureToPtr(managed, new IntPtr(pBuf), false);
    }

    //--- Attempt 2:
    //System.ArgumentException: 'Object contains non-primitive or non-blittable data'
    var managedHandle = GCHandle.Alloc(managed, GCHandleType.Pinned);
    Marshal.Copy(
        source: managedHandle.AddrOfPinnedObject(),
        destination: buffer,
        startIndex: 0, length: 12);
    managedHandle.Free();

}

Is there a way to marshal the Outer struct without me marshalling Outer_1, Outer_1 ... Outer_N by hand? Probably fine if I'm just dealing with the one, but there's tons of different messages I need to marshal.