C# ClientWebsocket throwing Exception on incoming binary message

I implemented a websocket-client using System.Net.WebSockets to communicate with an embedded device. On the device the libwebsocket library is in use.

The main part of the protocol is implemented using JSON strings, which works perfectly but some binary transmission is also needed. Outgoing binary message from Windows are received correctly on the device, but in the case of incoming binary messages a exception is thrown at ReceiveAsync().

I implemented the receive part as a asynchronous loop that calls ReceiveAsync(). In the case of an incoming binary message the WebSocketException with the following error code is thrown:

HResult 0x83760002

E_INVALID_PROTOCOL_FORMAT Protocol data had invalid format.

I don't know what causes it, it throws before i can look at the data i receive. It already worked to receive binary data from the device, but it was an early implementation and only looped back the data that i sent.

private async void ReceiveLoop(CancellationToken cancellationToken)
    {
        try
        {
            List<byte> receivedBytes = new List<byte>();
            var buffer = new byte[c_bufferSize];

            while (!cancellationToken.IsCancellationRequested)
            {
                var receiveBuffer = new ArraySegment<Byte>(buffer);
                WebSocketReceiveResult result;

                result = await _webSocket.ReceiveAsync(receiveBuffer, cancellationToken);

                receivedBytes.AddRange(receiveBuffer.Array);
                // message is complete, return it
                if (result.EndOfMessage)
                {
                    if (result.MessageType == WebSocketMessageType.Text)
                    {

                        String receivedString = Encoding.UTF8.GetString(receivedBytes.ToArray());
                        Debug.Print($"{receivedString}");
                        Debug.Print($"{receivedString.Length.ToString()}");
                        WebsocketMessage response = new WebsocketMessage(receivedString);
                        MessagesSubject.OnNext(response);
                    }

                    if (result.MessageType == WebSocketMessageType.Binary)
                    {
                        WebsocketMessage response = new WebsocketMessage(receivedBytes.ToArray());
                        MessagesSubject.OnNext(response);

                    }
                    receivedBytes.Clear();
                    buffer = new byte[c_bufferSize];
                }
            }
        }
        catch(OperationCanceledException ex)
        {
            return;
        }
        catch(Exception ex)
        {
            MessagesSubject.OnError(ex);
        }
    }