Server-side Blazor loses authentication state occassionally

I'm working on an server-side Blazor application which supports the "normal" build-in Identity login as well as AzureAD authentication (where the authenticated user is then mapped to a local one). The authentication and authorization themselves work fine. Logged in via build-in Identity I get an authentication cookie. Logged in via AzureAD I get the Azure cookies and the Identity cookie:

Authentication cookies

But occassionally the applications seems to hang, all cookies are removed and the authentication state is gone. I've got logged out. And I have no idea why this happens. The DevTools console then shows:

blazor.server.js:1 [2020-05-14T19:46:43.201Z] Information: Normalizing '_blazor' to 'https://wwwxxx/xxx/_blazor'.
blazor.server.js:1 [2020-05-14T19:56:56.047Z] Warning: Timeout from HTTP request.
blazor.server.js:1 GET https://wwwxxx/xxx/_blazor?id=cV_9FcIQ1C1raoggiSYzVg&_=1589486216048 404
blazor.server.js:1 [2020-05-14T19:56:56.559Z] Error: Connection disconnected with error 'Error'.
blazor.server.js:1 [2020-05-14T19:56:59.565Z] Information: Normalizing '_blazor' to 'https://wwwxxx/xxx/_blazor'.

Startup.cs looks as follows:

public void ConfigureServices(IServiceCollection services)
        {
            services.AddDbContext<ApplicationDbContext>(options =>
                options.UseSqlServer(
                    Configuration.GetConnectionString("DefaultConnection")));
            services.AddDefaultIdentity<AppUser>(options =>
                {
                    options.SignIn.RequireConfirmedAccount = true;
                    options.User.RequireUniqueEmail = true;
                })
                .AddRoles<IdentityRole>()
                .AddEntityFrameworkStores<ApplicationDbContext>()
                .AddDefaultTokenProviders();
            services.AddHttpContextAccessor();

            services.AddRazorPages().AddRazorPagesOptions(options =>
            {
               options.Conventions.AuthorizeFolder("/");
            }).AddMvcOptions(options =>
            {
                var policy = new AuthorizationPolicyBuilder()
                    .RequireAuthenticatedUser()
                    .Build();
                options.Filters.Add(new AuthorizeFilter(policy));
            }); ;
            services.AddServerSideBlazor();
            services.AddScoped<AuthenticationStateProvider, RevalidatingIdentityAuthenticationStateProvider<AppUser>>();

            services.AddAuthentication()
                .AddAzureAD(options => Configuration.Bind("AzureAd", options));
}

public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
        {
            //Localization
            app.UseRequestLocalization(app.ApplicationServices.GetService<IOptions<RequestLocalizationOptions>>().Value);

            if (env.IsDevelopment())
            {
                app.UseDeveloperExceptionPage();
                app.UseDatabaseErrorPage();
            }
            else
            {
                app.UseExceptionHandler("/Error");
                // The default HSTS value is 30 days. You may want to change this for production scenarios, see https://aka.ms/aspnetcore-hsts.
                app.UseHsts();
            }

            app.UseHttpsRedirection();
            app.UseStaticFiles();

            app.UseRouting();

            app.UseAuthentication();
            app.UseAuthorization();

            app.UseEndpoints(endpoints =>
            {
                endpoints.MapControllers();
                endpoints.MapBlazorHub();
                endpoints.MapFallbackToPage("/_Host");
            });
        }

Does anyone know this problem? Any suggestions? Thank you in advance!

Update:

It seems that the WebSocket connection breaks... I don't know why:

Microsoft.AspNetCore.SignalR.Internal.DefaultHubDispatcher: Debug: Received hub invocation: InvocationMessage { InvocationId: "", Target: "EndInvokeJSFromDotNet", Arguments: [ 1230, True, [1230,true,null] ], StreamIds: [  ] }.
Microsoft.AspNetCore.SignalR.Internal.DefaultHubDispatcher: Trace: InvocationId (null): Sending result of type 'System.Void'.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 2919 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 2919 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 23565 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 238 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 196 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 178 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 279 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 279 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 279 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 279 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 174 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 543 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 174 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 543 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 174 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 543 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 543 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 104 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 104 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 104 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 104 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 104 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 104 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 104 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 104 bytes.
Exception thrown: 'Microsoft.AspNetCore.Connections.ConnectionResetException' in Microsoft.AspNetCore.Server.IIS.dll
Exception thrown: 'Microsoft.AspNetCore.Connections.ConnectionResetException' in System.Private.CoreLib.dll
Exception thrown: 'Microsoft.AspNetCore.Connections.ConnectionResetException' in System.Private.CoreLib.dll
Exception thrown: 'Microsoft.AspNetCore.Connections.ConnectionResetException' in System.Private.CoreLib.dll
Exception thrown: 'Microsoft.AspNetCore.Connections.ConnectionResetException' in System.Private.CoreLib.dll
Exception thrown: 'System.Net.WebSockets.WebSocketException' in System.Net.WebSockets.dll
Exception thrown: 'System.Net.WebSockets.WebSocketException' in System.Private.CoreLib.dll
'iisexpress.exe' (CoreCLR: clrhost): Loaded 'C:\Program Files\dotnet\shared\Microsoft.NETCore.App\3.1.3\System.Diagnostics.StackTrace.dll'. 
'iisexpress.exe' (CoreCLR: clrhost): Loaded 'C:\Program Files\dotnet\shared\Microsoft.NETCore.App\3.1.3\System.Reflection.Metadata.dll'. 
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Debug: Socket connection closed prematurely.

System.Net.WebSockets.WebSocketException (0x80004005): The remote party closed the WebSocket connection without completing the close handshake.
 ---> Microsoft.AspNetCore.Connections.ConnectionResetException: The client has disconnected
 ---> System.Runtime.InteropServices.COMException (0x800704CD): Es wurde versucht, auf eine nicht vorhandene Netzwerkverbindung zuzugreifen. (0x800704CD)
   --- End of inner exception stack trace ---
   at Microsoft.AspNetCore.Server.IIS.Core.IO.AsyncIOOperation.GetResult(Int16 token)
   at Microsoft.AspNetCore.Server.IIS.Core.IISHttpContext.ReadBody()
   at System.IO.Pipelines.PipeCompletion.ThrowLatchedException()
   at System.IO.Pipelines.Pipe.GetReadResult(ReadResult& result)
   at System.IO.Pipelines.Pipe.GetReadAsyncResult()
   at Microsoft.AspNetCore.Server.IIS.Core.IISHttpContext.ReadAsync(Memory`1 memory, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Server.IIS.Core.HttpRequestStream.ReadAsyncInternal(Memory`1 buffer, CancellationToken cancellationToken)
   at System.Net.WebSockets.ManagedWebSocket.EnsureBufferContainsAsync(Int32 minimumRequiredBytes, CancellationToken cancellationToken, Boolean throwOnPrematureClosure)
   at System.Net.WebSockets.ManagedWebSocket.ReceiveAsyncPrivate[TWebSocketReceiveResultGetter,TWebSocketReceiveResult](Memory`1 payloadBuffer, CancellationToken cancellationToken, TWebSocketReceiveResultGetter resultGetter)
   at System.Net.WebSockets.ManagedWebSocket.ReceiveAsyncPrivate[TWebSocketReceiveResultGetter,TWebSocketReceiveResult](Memory`1 payloadBuffer, CancellationToken cancellationToken, TWebSocketReceiveResultGetter resultGetter)
   at Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsServerTransport.StartReceiving(WebSocket socket)
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Debug: Waiting for the application to finish sending data.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Debug: Socket closed.
Microsoft.AspNetCore.Http.Connections.Internal.HttpConnectionContext: Trace: Disposing connection 8FE-SmJEB1D0RUWx8dXjLA.
Microsoft.AspNetCore.Http.Connections.Internal.HttpConnectionContext: Trace: Waiting for application to complete.
Microsoft.AspNetCore.SignalR.HubConnectionHandler: Debug: OnConnectedAsync ending.
Microsoft.AspNetCore.Http.Connections.Internal.HttpConnectionContext: Trace: Application complete.
Microsoft.AspNetCore.Http.Connections.Internal.HttpConnectionManager: Debug: Removing connection qSMJEL5tNU3rTLk5nrX_xw from the list of connections.
Microsoft.AspNetCore.Http.Connections.Internal.HttpConnectionManager: Debug: New connection es3QXXbAgihjUVXi211hVw created.
Microsoft.AspNetCore.Http.Connections.Internal.HttpConnectionDispatcher: Debug: Sending negotiation response.
Microsoft.AspNetCore.Http.Connections.Internal.HttpConnectionDispatcher: Debug: Establishing new connection.
Microsoft.AspNetCore.SignalR.HubConnectionHandler: Debug: OnConnectedAsync started.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Debug: Socket opened using Sub-Protocol: '(null)'.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Message received. Type: Text, size: 38, EndOfMessage: True.
Microsoft.AspNetCore.SignalR.Internal.DefaultHubProtocolResolver: Debug: Found protocol implementation for requested protocol: blazorpack.
Microsoft.AspNetCore.SignalR.HubConnectionContext: Debug: Completed connection handshake. Using HubProtocol 'blazorpack'.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 3 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Message received. Type: Binary, size: 245, EndOfMessage: True.
Microsoft.AspNetCore.SignalR.Internal.DefaultHubDispatcher: Debug: Received hub invocation: InvocationMessage { InvocationId: "0", Target: "ConnectCircuit", Arguments: [ CfDJ8Hy6nGrT4bRAtvHZDDv3QE4idYpgvocWjooUF0ypIU4d2QFQXnLVnBfBIzseKV-LRhC60wxVjdqZkcUdFGKUASZUiIU4a5mWs2TYam17ichgx6gxoM5ueM7_v9JuiXbr91zEbuiw0MHdHeCb6U_QPCA_36siQCR0h3BfgEwTdqEjdB0PwG7CROaiqdapc-obGR9-vIeRtB0GjsLrNSJy2N0 ], StreamIds: [  ] }.
Exception thrown: 'System.Threading.Tasks.TaskCanceledException' in System.Private.CoreLib.dll
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 18869 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 2919 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 48272 bytes.
Microsoft.AspNetCore.SignalR.Internal.DefaultHubDispatcher: Trace: InvocationId 0: Sending result of type 'System.Boolean'.