multiple websocket disconnection from signalr cause application pool freeze

0

We have a asp.net website developed in .net framework 4.6 deployed in iis. We are using Signalr 2.2 in order to notify clients in real time. We generally have 500 clients connected. Transport used is Websocket. Sometimes happen that we have a lot of connection aborted at the same time with this error:

11:35:49.759| |ERROR| |ProcessId=17672| |ThreadId=702| |SignalR.Transports.WebSocketTransport| |OnError(e42c532d-d3d1-4649-a8c7-3d7810b2a74e, System.Net.WebSockets.WebSocketException (0x800703E3): The I/O operation has been aborted because of either a thread exit or an application request
   at System.Web.WebSockets.WebSocketPipe.<>c__DisplayClass9_0.<ReadFragmentAsync>b__0(Int32 hrError, Int32 cbIO, Boolean fUtf8Encoded, Boolean fFinalFragment, Boolean fClose)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Web.WebSockets.AspNetWebSocket.<DoWork>d__45`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Web.WebSockets.AspNetWebSocket.<>c__DisplayClass36_0.<<ReceiveAsyncImpl>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.AspNet.SignalR.WebSockets.WebSocketMessageReader.<ReadMessageAsync>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.AspNet.SignalR.WebSockets.WebSocketHandler.<ProcessWebSocketRequestAsync>d__25.MoveNext())| |

The problem is that during this multiple disconnection the application pool seems to be "freezed". Any request go in timeout and we have a lot of this errors:

11:36:32.762| |DEBUG| |ProcessId=17672| |ThreadId=1440| |Infrastructure.ExceptionHandling.ExceptionFilterAttributeHandler| |Wrapping application exception A task was canceled.| |
11:36:32.762| |ERROR| |ProcessId=17672| |ThreadId=1440| |WebApiTraceWriter| |Kind=End, Category=System.Web.Http.Filters, Message='', Operation=OnExceptionAsync, Operator=ExceptionFilterAttributeHandler, Exception=A task was canceled.| |

Sometimes it came back to work after a lot of minutes, sometimes instead it was necessary to restart the application pool in order to solve the problem.

UPDATE I noticed that the real disconnection of clients, happens about 15 minutes after the aborted connection:

        public override Task OnDisconnected(bool stopCalled)
        {
            string connId = Context.ConnectionId;
            _log.InfoFormat("FOR_FILE: SignalR Server - ConnectionId [{0}] - OnDisconnected with stopCalled = {1}", connId, stopCalled.ToString());

            return base.OnDisconnected(stopCalled);
        }

When it happens the application come back to work normally. But why after 15 minutes? Is possible to configure this value?

.net
iis
websocket
signalr
application-pool
asked on Stack Overflow May 20, 2021 by Angelo • edited May 21, 2021 by Angelo

1 Answer

0

SOLVED

I have solved the problem removing the Signalr diagnostic part in the Web.Config.

In production environment with a lot of clients connected, in case of network problem, it cause that all the connections are not killed immediately but after few minutes or after the recycle of the application pool. During these minutes the application pool was blocked and did not process anything.

answered on Stack Overflow May 28, 2021 by Angelo

User contributions licensed under CC BY-SA 3.0