Memory allocation failed on sending Large byte[] using WCF in buffered mode

I want to use WCF for transfer file between clients and server. I read many article that some people using Stream mode for this purpose. but for it's restrictions I decide to create service with buffered mode. For example consider this:

[OperationContract]
void UploadFile(Guid systemKey, string fileName, byte[] fileContent, string userName);

server web config:

...
<system.web>
    <compilation debug="true" targetFramework="4.6" />
    <httpRuntime targetFramework="4.6" maxRequestLength="2147483647" executionTimeout="7200"/>
</system.web>
....
<bindings>
  <basicHttpBinding>
    <binding name="myBasicBinding" maxBufferSize="2147483647" 
                                   maxReceivedMessageSize="2147483647" 
                                   closeTimeout="01:50:00" 
                                   openTimeout="01:50:00" 
                                   sendTimeout="01:50:00" 
                                   receiveTimeout="01:50:00" 
                                   messageEncoding="Mtom">
      <readerQuotas maxDepth="128" maxStringContentLength="8388608" maxArrayLength="2147483647" maxBytesPerRead="4096" maxNameTableCharCount="16384" />
    </binding>
  </basicHttpBinding>
</bindings>
....
<system.webServer>
   <security>
      <requestFiltering>
           <requestLimits maxAllowedContentLength="4294967295"/>
      </requestFiltering>
   </security>
   <modules runAllManagedModulesForAllRequests="true"/>
   <directoryBrowse enabled="true"/>
</system.webServer>

and client config:

<system.web>
    <compilation debug="true" targetFramework="4.6"/>
    <httpRuntime targetFramework="4.6" maxRequestLength="2147483647" executionTimeout="7200"/>
</system.web>
...
<system.webServer>
    <security>
      <requestFiltering>
        <requestLimits maxAllowedContentLength="4294967295"/>
      </requestFiltering>
    </security>
</system.webServer>
<system.serviceModel>
    <bindings>
        <basicHttpBinding>
        <binding name="basicEndpoint"
                 maxBufferSize="2147483647"
                 maxReceivedMessageSize="2147483647"
                 closeTimeout="01:50:00"
                 openTimeout="01:50:00"
                 sendTimeout="01:50:00"
                 receiveTimeout="01:50:00"
                 messageEncoding="Mtom">
          <readerQuotas maxDepth="128" maxStringContentLength="8388608" maxArrayLength="2147483647" maxBytesPerRead="4096" maxNameTableCharCount="16384" />
       </binding>
     </basicHttpBinding>
   </bindings>
   <client>
      <endpoint address="http://localhost/fts.svc" binding="basicHttpBinding" bindingConfiguration="basicEndpoint" contract="ServiceReference1.Ifts" name="basicEndpoint"/>
   </client>
</system.serviceModel>

I can upload 400MB file but when the size of file is 500MB or above I got this error:

System.InsufficientMemoryException HResult=0x8013153D Message=Failed to allocate a managed memory buffer of 1073741824 bytes. The amount of available memory may be low. Source=mscorlib

Inner Exception 1: OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown. \

I think I set values for 2GB data but it fails on 500MB.

Thanks

this code run locally and I have 16GB RAM. Where is the problem and what value I should change to upload harge files?

Thanks

I remember I was working with a large file transfer, and I used buffered mode to transfer but was not the best solution because after a while I got some exception about memory leak or something related memory, I changed it to transferMode="Streamed" and everything worked fine, by the buffered mode it means that the entire content of a message is present in memory before it is sent or after it is received. While that is a good strategy for most scenarios, and necessary for messaging features such as digital signatures and reliable delivery, large messages could exhaust a system's resources.
take a look at here.

To avoid creating new buffers all the time WCF uses BufferManager to reuse buffers, up to the limit specified by maxBufferPoolSize, buffers are expensive to create and destroy. You can use the BufferManager class to manage a buffer pool.

You can try increasing maxBufferPoolSize to see if you can reduce memory usage. I strongly recommend do not increase it to max because I think buffers from the pool are never released until the app-domain get recycles. A period of high traffic could cause a lot of memory to be used and never released.

So I recommend you to use streamed mode to transfer large file.

To split files and works with large streams below links may help you:

Multiple file in one Stream, custom stream
Best way to return large file as split zip files,Stream or Byte array WCF

it may also help you

Multiple stream in one stream will not passed to client properly