File download HTTPHandler throwing error on HttpResponse.Flush

I have an HTTPHandler which sends a file to the client. I am seeing errors being logged only sometimes. The file being downloaded in all cases is pretty small, less than 1MB. Here is the error/stack trace:

The remote host closed the connection. The error code is 0x800703E3.

at System.Web.Hosting.IIS7WorkerRequest.RaiseCommunicationError(Int32 result, Boolean throwOnDisconnect)
at System.Web.Hosting.IIS7WorkerRequest.ExplicitFlush() at System.Web.HttpResponse.Flush(Boolean finalFlush)

And here is the code:

public class DownloadHttpHandler : IHttpHandler
{
    public bool IsReusable { get { return true; } }

    public void ProcessRequest(HttpContext context)
    {
        //Checking permission, getting the file path, etc...

        ResponseUtil.SendDownloadFile(context.Response, fullPath);
    }
}

public static class ResponseUtil
{
    /// <summary>Sends the specified file.</summary>
    public static void SendDownloadFile(HttpResponse response, string path, string contentType)
    {
        FileInfo fileInfo = new FileInfo(path);

        BeginSendDownloadFile(response, fileInfo.Name, contentType, fileInfo.Length);

        using (FileStream stream = File.OpenRead(path))
        {
            stream.CopyTo(response.OutputStream);
        }

        EndSendDownloadFile(response);
    }

    /// <summary>Prepares the output stream to send a downloadable file.</summary>
    public static void BeginSendDownloadFile(HttpResponse response, string filename, string contentType, long contentLength)
    {
        if (response.IsClientConnected)
        {
            response.AddHeader("Content-Disposition", "attachment; filename={0}".FormatString(filename));
            response.ContentType = contentType;
            response.AddHeader("Content-Length", contentLength.ToString());
        }
    }

    /// <summary>Flushes and closes the output stream.</summary>
    public static void EndSendDownloadFile(HttpResponse response)
    {
        if (response.IsClientConnected)
        {
            response.Flush();
            response.Close();
        }
    }
}

I thought maybe the download was being cancelled so I added the response.IsClientConnected checks in a couple of spots. But I'm still seeing the error.

Should I just not call Flush and Close?