IIS ASP.Net Website - ManagedPipelineHandler Error ( An operation was attempted on a nonexistent network connection)

I am experiencing a strange issue. I have an ASP.Net web application that uses Forms Authentication. If I open the log-in page and there is a couple of minutes of inactivity, the log-in attempt times out or IE / FireFox display "Cannot display this page". If there is no pause or lack of activity, there is no issue and the website run as expected.

I enabled tracing in IIS. The area that appears to be hanging is NOTIFY_MODULE_START Module Name:ManagedPipelineHandler. After 132141 ms, I get the following error The I/O operation has been aborted because of either a thread exit or an application request. (0x800703e3) - ErrorCode 2147943395.

Error Code 2147943629

Error Code An operation was attempted on a nonexistent network connection. (0x800704cd)

Screen Shot: http://www.plasmawebservices.com/images/errorScreenShot.png

Server is Windows 2008 SP2. IIS 7

Has anyone come across an error like this?

<asp:Content ID="BodyContent" runat="server" ContentPlaceHolderID="MainContent">
<telerik:RadFormDecorator ID="RadFormDecorator" runat="server" DecoratedControls="All" />
<p>
    Please enter your username and password.
</p>
<asp:Login ID="LoginUser" runat="server" EnableViewState="false" RenderOuterTable="false">
    <LayoutTemplate>
        <asp:Panel runat="server" ID="LoginPanel" DefaultButton="LoginButton">
            <p>
                <span class="failure-notification">
                    <asp:Literal ID="FailureText" runat="server"></asp:Literal>
                </span>
            </p>
            <asp:ValidationSummary ID="LoginUserValidationSummary" runat="server" CssClass="failure-notification"
                ValidationGroup="LoginUserValidationGroup" />
            <div class="account-info">
                <fieldset class="login">
                    <legend>Account Information</legend>
                    <p>
                        <asp:Label ID="UserNameLabel" runat="server" AssociatedControlID="UserName">Username:</asp:Label>
                        <asp:TextBox ID="UserName" runat="server" CssClass="text-entry"></asp:TextBox>
                        <asp:RequiredFieldValidator ID="UserNameRequired" runat="server" ControlToValidate="UserName"
                            CssClass="failure-notification" ErrorMessage="User Name is required." ToolTip="User Name is required."
                            ValidationGroup="LoginUserValidationGroup">*</asp:RequiredFieldValidator>
                    </p>
                    <p>
                        <asp:Label ID="PasswordLabel" runat="server" AssociatedControlID="Password">Password:</asp:Label>
                        <asp:TextBox ID="Password" runat="server" CssClass="password-entry" TextMode="Password"></asp:TextBox>
                        <asp:RequiredFieldValidator ID="PasswordRequired" runat="server" ControlToValidate="Password"
                            CssClass="failure-notification" ErrorMessage="Password is required." ToolTip="Password is required."
                            ValidationGroup="LoginUserValidationGroup">*</asp:RequiredFieldValidator>
                    </p>
                    <%--<p>
                        <asp:CheckBox ID="RememberMe" runat="server"/>
                    <asp:Label ID="RememberMeLabel" runat="server" AssociatedControlID="RememberMe" CssClass="inline">Keep me logged in</asp:Label>
                    </p>--%>
                </fieldset>
                <p class="margin-top-10">
                    <asp:Button ID="LoginButton" runat="server" CommandName="Login" Text="Log In" ValidationGroup="LoginUserValidationGroup" />
                </p>
            </div>
        </asp:Panel>
    </LayoutTemplate>
</asp:Login>

Thanks, Brennan

Solution

The client had AVG Security workstation components installed on their servers. The workstation components caused communication issues with IIS. By disabling, Online Shield and Resident Shield the symptoms are no longer present.

Case Notes:

After two minutes of inactivity on any page on the web application, the page locks. You will see an Ajax loading panel ( aka spinning icon) for 120 seconds on your next action and then nothing. The application has locked.

To find out the root cause as to why we are getting the error "An operation was attempted on a nonexistent network connection. (0x800704cd)" when we browse any web page after an idle time of minutes. This issue can occur due to multiple reasons and for the purpose of this ticket we will concentrate on only one root cause If the issue is caused by a Microsoft component, we will work diligently and efficiently to resolve this issue. If the cause of the issue is identified to be a third party or custom component, we will be happy to discuss our findings in detail and provide possible methods of resolution/workaround. It will ultimately be up to the third party vendor to resolve these types of issues.

For troubleshooting, the default ASP.Net website was created and deployed as a new website to IIS. The results were the same as the problematic website.

IIS HTTP Errors:

start -> run -> LogFiles

2011-09-20 21:25:25 xx.xxx.xxx.xxx 36482 192.168.10.32 80 HTTP/1.1 POST /orders/mail/mailorderentry - - Timer_HeaderWait –

2011-09-20 21:22:29 xx.xxx.xxx.xxx 44399 192.168.10.32 80 HTTP/1.1 POST /orders/mail/mailorderentry - - Client_Reset –

At first, the issue apperared to be with Telrik. While loading up the Telerik.Web.UI.RadCompression on the REQUEST_ACQUIRE_STATE event, the following error was logged via IIS tracing:

Telerik.Web.UI.RadCompression 132187 ms

1. -GENERAL_READ_ENTITY_END BytesReceived 0 ErrorCode 2147943395 ErrorCode The I/O operation has been aborted because of either a thread exit or an application request. (0x800703e3) Informational

2. -GENERAL_FLUSH_RESPONSE_END BytesSent 0 ErrorCode 2147943629 ErrorCode An operation was attempted on a nonexistent network connection. 0x800704cd)

The RadCompression module was removed from the web application web.config file; however, the problem persisted.

Proposed Solution:

For now disable, AVG Online Shield and Resident Shield on the servers.

From AVG:

The following AVG components are designed only for a use on workstations:

• AVG Firewall
• AVG Online Shield
• AVG Personal E-mail Scanner
• AVG LinkScanner

The above components were not tested under a system load. If installed on a server operating system (such as ISA, proxy, mail server, terminal server, etc.) the components might cause issues with server communication. To avoid these issues it is not recommended to install the mentioned components on a server operating system. Note: This does not apply in case that you use the computer with some server operating system installed (e.g. Windows Server 2000, 2003 etc.) as a standard computer without the server features enabled.

More information about the AVG installation on specific servers can be found in these FAQs:

• FAQ 2129 - Terminal Server
• FAQ 2176 - E-mail Server