"An existing connection was forcibly closed by the remote host" during file upload to Azure Blob Storage

2

I am trying to upload multiple files to Azure Storage asynchronously using Task.WhenAll(tasks);

However, I keep running into the following error

Microsoft.WindowsAzure.Storage.StorageException
  HResult=0x80131500
  Message=Error while copying content to a stream.
  Source=Microsoft.WindowsAzure.Storage
  StackTrace:
   at Microsoft.WindowsAzure.Storage.Core.Executor.Executor.<ExecuteAsyncInternal>d__4`1.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.WindowsAzure.Storage.Blob.CloudBlockBlob.<UploadFromStreamAsyncHelper>d__34.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.WindowsAzure.Storage.Blob.CloudBlockBlob.<UploadFromStreamAsyncHelper>d__33.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at RetsIntegration.Services.AzureBlobStorage.<CreateAsync>d__9.MoveNext() 

Inner Exception 1:
HttpRequestException: Error while copying content to a stream.

Inner Exception 2:
IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.

Inner Exception 3:
SocketException: An existing connection was forcibly closed by the remote host

The error happens after a couple of minutes of executing the tasks.

I create multiple tasks to upload images using the following logic

public async Task ProcessTasks(IEnumerable<FileObject> files)
{
    List<Task> tasks = new List<Task>();

    foreach (FileObject file in files)
    {
        string path = GetImageFullName(file, file.Filename);

        Task task Storage.CreateAsync(file.Content, path);

        tasks.Add(task);
    }

    // begings the upload to Azure
    await Task.WhenAll(tasks);
}

Here is the implementation of the CreateAsync method from the Storage class

public async Task CreateAsync(Stream stream, string path)
{
    var azure = GetAzurePath(path);
    // ContainerFactory is IBlobContainerFactory type
    var container = await ContainerFactory.GetAsync();

    CloudBlockBlob blockBlob = container.GetBlockBlobReference(azure);

    await blockBlob.UploadFromStreamAsync(stream);
}

Here is how I create the Blob containers. Note when I create the container I increase the server timeout to 4 hours which should be plenty of time to complete the upload.

public class DefaultBlobContainerFactory : IBlobContainerFactory
{
    private readonly CloudBlobContainer _container;

    public DefaultBlobContainerFactory(AzureBlobOptions azureBlobOptions)
    {
        try
        {
            CloudBlobClient blobClient = GetClient(azureBlobOptions);
            _container = blobClient.GetContainerReference(azureBlobOptions.DocumentContainer);
        }
        catch (StorageException)
        {
            throw;
        }
    }

    protected virtual CloudBlobClient GetClient(AzureBlobOptions azureBlobOptions)
    {
        if (azureBlobOptions.ConnectionString != null && CloudStorageAccount.TryParse(azureBlobOptions.ConnectionString, out CloudStorageAccount cloudStorageAccount))
        {
            return cloudStorageAccount.CreateCloudBlobClient();
        }

        if (azureBlobOptions.BaseUri != null && azureBlobOptions.Token != null)
        {
            return new CloudBlobClient(azureBlobOptions.BaseUri, new StorageCredentials(azureBlobOptions.Token));
        }

        throw new ArgumentException("One of the following must be set: 'ConnectionString' or 'BaseUri'+'Token'!");
    }

    public async Task<CloudBlobContainer> GetAsync()
    {
        await _container.CreateIfNotExistsAsync(new BlobRequestOptions()
        {
            ServerTimeout = TimeSpan.FromHours(4),
        }, null);

        return _container;
    }
}

What could be causing this issue? How can I fix it?

azure
azure-storage
azure-storage-blobs
blobstorage
asked on Stack Overflow May 12, 2019 by Junior • edited May 12, 2019 by Junior

0 Answers

Nobody has answered this question yet.


User contributions licensed under CC BY-SA 3.0