I have problems with streaming large file upload in ASP.NET 4.7.
I'm using custom WebHostBufferPolicySelector
to disable buffering of input stream (as described here: https://www.strathweb.com/2012/09/dealing-with-large-files-in-asp-net-web-api/) and I'm sending file in HTTP POST request with multipart/form-data
. I read data in controller using HttpContext.Current.Request.Files["key"].InputStream
. I'm using it with FileStream
to write uploaded data to file on server. Basically I use InputStream.CopyToAsync
method with 80kB buffer size (this seems to be the default value) to copy data.
Edit: Added code example
public class FileUploadChunkResBindingModel
{
public string Status { get; set; }
public string FileName { get; set; }
public string Guid { get; set; }
}
[RoutePrefix("api/FileUpload")]
public class FileUploadController : ApiController
{
private ILogger<FileUploadController > _logger;
[HttpPost]
[Route("UploadFile")]
[Authorize(Roles = "WriteAdmin")]
public async Task<FileUploadChunkResBindingModel> UploadFileAsync(CancellationToken ct)
{
var req = HttpContext.Current.Request;
if (req.Files.Count != 1)
{
throw new HttpResponseException(new HttpResponseMessage
{
StatusCode = HttpStatusCode.BadRequest,
Content = new StringContent(string.Format("Request must contain 1 file, but contains {0}", req.Files.Count)),
});
}
var guid = req.Form["guid"];
var file = req.Files[req.Files.Keys.Get(0)];
var ret = await UploadFileAsync(guid, file, ct);
return ret;
}
public async Task<FileUploadChunkResBindingModel> UploadFileAsync(string guid, HttpPostedFile file, CancellationToken ct = default)
{
var path = Path.Combine("~", "FileUpload", guid);
var uploadDir = HostingEnvironment.MapPath(path);
Directory.CreateDirectory(uploadDir);
try
{
var filePath = Path.Combine(uploadDir, file.FileName);
using (var fileReadStream = file.InputStream)
using (var fileWriteStream = File.OpenWrite(filePath))
{
var bufferSize = 81920; // 80kB should be default size based on documentation
await fileReadStream.CopyToAsync(fileWriteStream, bufferSize, ct);
}
return new FileUploadChunkResBindingModel
{
Status = "Done",
Guid = guid,
FileName = file.FileName
};
}
catch (Exception exc)
{
_logger.LogError(exc, "File upload failed with exception.");
Directory.Delete(uploadDir, true);
return new FileUploadChunkResBindingModel
{
Status = "Cancelled"
};
}
}
}
This is code for WebHostBufferPolicySelector
:
public class NoBufferPolicySelector : WebHostBufferPolicySelector
{
public override bool UseBufferedInputStream(object hostContext)
{
if (hostContext is HttpContextBase context)
{
if (context.Request.CurrentExecutionFilePath.EndsWith("UploadFile"))
{
// Do not use buffered input stream for file uploads - this reduces memory footprint
return false;
}
}
return true;
}
}
And it is registered in Global.asax.cs
public class Global : HttpApplication
{
void Application_Start(object sender, EventArgs e)
{
GlobalConfiguration.Configuration.Services.Replace(typeof(IHostBufferPolicySelector), new NoBufferPolicySelector());
}
}
(it's little bit simplified code example, I hope I didn't remove any important part for this issue).
This seems to work for HTTP 1.1 but as soon I switch to HTTP 2, I start getting following exceptions:
System.Web.HttpException (0x80004005): An error occurred while communicating with the remote host. The error code is 0x800703E3. ---> System.Runtime.InteropServices.COMException (0x800703E3): The I/O operation has been aborted because of either a thread exit or an application request. (Exception from HRESULT: 0x800703E3)
at System.Web.Hosting.IIS7WorkerRequest.RaiseCommunicationError(Int32 result, Boolean throwOnDisconnect)
at System.Web.Hosting.IIS7WorkerRequest.ReadEntityCoreSync(Byte[] buffer, Int32 offset, Int32 size)
at System.Web.Hosting.IIS7WorkerRequest.ReadEntityBody(Byte[] buffer, Int32 size)
at System.Web.HttpRequest.GetEntireRawContent()
at System.Web.HttpRequest.GetMultipartContent()
at System.Web.HttpRequest.FillInFilesCollection()
at System.Web.HttpRequest.EnsureFiles()
at System.Web.HttpRequest.get_Files()
at Lic.Web.Controllers.FileUploadController.<UploadFileAsync>d__6.MoveNext() in C:\projects\emclient\license-manager\Lic.Web\Controllers\FileUploadController.cs:line 93
sometimes I get this exception after just few first megabytes of files are transfered.
Is this the right approach for large file upload? Basically I want to avoid loading whole file into memory, I'd like to use streams to process it with small buffer to avoid memory congestion problems.
I know there are similar posts with related problems, but I think none of them solves this problem for HTTP2. And also, most of them are quite old, so maybe there is some better approach in newer versions of ASP.NET.
User contributions licensed under CC BY-SA 3.0