sporadic An error occurred while receiving the HTTP response to https://www.mydomain/service.svc

0

I've googled around and read different answers on stackoverflow and I"m still lost. I have a server that has multiple WCF services running. The Services call each other for example

A -> B -> C

Most of the time the services work without issue but every now and then I'll get the error message. It could happen calling service A. Or the issue could happen with service A calling Service B, or with service B calling service C.

when I follow the WCF Diagnostics logs I can see successful calls between service by looking for the Message ID: uuid. Message ID

however when the call fails the client has the following stack trace. (highlights only)

<E2ETraceEvent xmlns="http://schemas.microsoft.com/2004/06/E2ETraceEvent">
    <System xmlns="http://schemas.microsoft.com/2004/06/windows/eventlog/system">
        <EventID>131075</EventID>
        <Type>3</Type>
        <SubType Name="Error">0</SubType>
        <Level>2</Level>
        <TimeCreated SystemTime="2020-08-19T23:36:08.2460729Z" />
        <Source Name="System.ServiceModel" />
        <Correlation ActivityID="{4654b130-00e0-4989-aee2-326fa56ab744}" />
        <Execution ProcessName="w3wp" ProcessID="7700" ThreadID="12" />
        <Channel />
        <Computer>MyComptureName</Computer>
    </System>
    <ApplicationData>
        <TraceData>
            <DataItem>
                <TraceRecord xmlns="http://schemas.microsoft.com/2004/10/E2ETraceEvent/TraceRecord" Severity="Error">
                    <TraceIdentifier>http://msdn.microsoft.com/en-US/library/System.ServiceModel.Diagnostics.ThrowingException.aspx</TraceIdentifier>
                    <Description>Throwing an exception.</Description>
                    <AppDomain>/LM/W3SVC/2/ROOT/MyAppDomain</AppDomain>
                    <Exception>
                        <ExceptionType>System.ServiceModel.CommunicationException, System.ServiceModel, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</ExceptionType>
                        <Message>An error occurred while receiving the HTTP response to https://www.example.com/Service.svc. This could be due to the service endpoint binding not using the HTTP protocol. This could also be due to an HTTP request context being aborted by the server (possibly due to the service shutting down). See server logs for more details.</Message>
                        <StackTrace>
                            ..lots of stackTrace info
                        </StackTrace>
                        <ExceptionString>System.ServiceModel.CommunicationException: An error occurred while receiving the HTTP response to  https://www.example.com/Service.svc. This could be due to the service endpoint binding not using the HTTP protocol. This could also be due to an HTTP request context being aborted by the server (possibly due to the service shutting down). See server logs for more details. ---&gt; System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive. ---&gt; System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---&gt; System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
   at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   --- End of inner exception stack trace ---
   
   --- End of inner exception stack trace ---
   
   --- End of inner exception stack trace --- </ExceptionString>
                        <InnerException>
                            <ExceptionType>System.Net.WebException, System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</ExceptionType>
                            <Message>The underlying connection was closed: An unexpected error occurred on a receive.</Message>
                            <StackTrace>
                               ..stackTrace info
                            </StackTrace>
                            <ExceptionString>System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive. ---&gt; System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---&gt; System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
   at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   --- End of inner exception stack trace ---
   at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   at System.Net.FixedSizeReader.ReadPacket(Byte[] buffer, Int32 offset, Int32 count)
   at System.Net.Security._SslStream.StartFrameHeader(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security._SslStream.StartReading(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security._SslStream.ProcessRead(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.TlsStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   at System.Net.PooledStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   at System.Net.Connection.SyncRead(HttpWebRequest request, Boolean userRetrievedStream, Boolean probeRead)
   --- End of inner exception stack trace ---
   at System.Net.HttpWebRequest.GetResponse()
   at System.ServiceModel.Channels.HttpChannelFactory`1.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout)</ExceptionString>
                            <InnerException>
                                <ExceptionType>System.IO.IOException, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</ExceptionType>
                                <Message>Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.</Message>
                                <StackTrace>

                                </StackTrace>
                                <ExceptionString>System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---&gt; System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
  
                                <InnerException>
                                    <ExceptionType>System.Net.Sockets.SocketException, System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</ExceptionType>
                                    <Message>An existing connection was forcibly closed by the remote host</Message>
                                    <StackTrace>
at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
                                    </StackTrace>
                                    <ExceptionString>System.Net.Sockets.SocketException (0x80004005): An existing connection was forcibly closed by the remote host
   at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)</ExceptionString>
                                    <NativeErrorCode>2746</NativeErrorCode>
                                </InnerException>
                            </InnerException>
                        </InnerException>
                    </Exception>
                </TraceRecord>
            </DataItem>
        </TraceData>
    </ApplicationData>
</E2ETraceEvent>

on the server side I don't see anything. It looks like the call did not even connect. It's not like the successful calls were I can see the call arrive to the service in the logs. The call only takes a few MS so there is no time out.

If I keep sending the same message eventually it will work. during the time period in which the call fails all other calls also fail. once its working again all the calls the previously failed start working.

I can't imagine this is a network issue since all the services are on the same server.

looking for any ideas that might help.

c#
wcf

0 Answers

Nobody has answered this question yet.


User contributions licensed under CC BY-SA 3.0