Reducing memory usage of .NET applications?

1. You might want to check out Stack Overflow question .NET EXE memory footprint.
2. The MSDN blog post Working set != actual memory footprint is all about demystifying the working set, process memory and how to perform accurate calculations on your total in-RAM consumption.

I will not say that you should ignore the memory footprint of your application -- obviously, smaller and more efficient does tend to be desirable. However, you should consider what your actual needs are.

If you are writing a standard Windows Forms and WPF client applications which is destined to run on an individual's PC, and is likely to be the primary application in which the user operates, you can get away with being more lackadaisical about memory allocation. (So long as it all gets deallocated.)

However, to address some folks here who say not to worry about it: If you're writing a Windows Forms application which will be running in a terminal services environment, on a shared server possibly utilized by 10, 20 or more users, then yes, you absolutely must consider memory usage. And you will need to be vigilant. The best way to address this is with good data structure design and by following best practices regarding when and what you allocate.

One thing you need to consider in this case is the memory cost of the CLR. The CLR is loaded for every .Net process and hence factors into the memory considerations. For such a simple / small program the cost of the CLR is going to dominate your memory footprint.

It would be much more instructive to construct a real application and view the cost of that compared to the cost of this baseline program.

No specific suggestions per se, but you might take a look at the CLR Profiler (free download from Microsoft).
Once you've installed it, take a look at this how-to page.

From the how-to:

This How To shows you how to use the CLR Profiler tool to investigate your application's memory allocation profile. You can use CLR Profiler to identify code that causes memory problems, such as memory leaks and excessive or inefficient garbage collection.

Might want to look at the memory usage of a "real" application.

Similar to Java there is some fixed amount of overhead for the runtime regardless of the program size, but memory consumption will be much more reasonable after that point.

There are still ways to reduce the private working set of this simple program:

1. NGEN your application. This removes JIT compilation cost from your process.

2. Train your application using MPGO reducing memory usage and then NGEN it.

There are many ways to reduce your footprint.

One thing you'll always have to live with in .NET is that the size of the native image of your IL code is huge

And this code cannot be completely shared between application instances. Even NGEN'ed assemblies are not completely static, they have still some little parts that need JITting.

People also tend to write code that blocks memory far longer than necessary.

An often seen example: Taking a Datareader, loading the contents into a DataTable just to write it into an XML file. You can easily run into an OutOfMemoryException. OTOH, you could use an XmlTextWriter and scroll through the Datareader, emitting XmlNodes as you scroll through the database cursor. That way, you only have the current database record and its XML output in memory. Which will never (or is unlikely to) get a higher garbage collection generation and thus can be reused.

The same applies to getting a list of some instances, doing some stuff (that spawns of thousands of new instances, which might stay referenced somewhere), and even though you don't need them afterwards, you still reference everything until after the foreach. Explicitly null-ing your input list and your temporary by-products means, this memory can be reused even before you exit your loop.

C# has an excellent feature called iterators. They allow you to stream objects by scrolling through your input and only keep the current instance until you get the next one. Even by using LINQ, you still don't need to keep all of it around just because you wanted it to be filtered.

Addressing the general question in the title and not the specific question:

If you are using a COM component that returns a lot of data (say large 2xN arrays of doubles) and only a small fraction is needed then one can write a wrapper COM component that hides the memory from .NET and returning only the data that is needed.

That is what I did in my main application and it significantly improved the memory consumption.

I have found that using the SetProcessWorkingSetSize or EmptyWorkingSet APIs to force memory pages to disk periodically in a long running process can result in all available physical memory on a machine to effectively disappear until the machine is rebooted. We had a .NET DLL loaded into a native process which would use the EmptyWorkingSet API (an alternative to using SetProcessWorkingSetSize) to reduce the working set after performing a memory intensive task. I found that after anywhere between 1 day to a week a machine would show 99% physical memory usage in Task Manager while no processes were shown to be using any significant memory usage. Soon after the machine would become unresponsive, requiring a hard reboot. Said machines were over 2 dozen Windows Server 2008 R2 and 2012 R2 servers running on both physical and virtual hardware.

Perhaps having the .NET code loaded into a native process had something to do with it, but use EmptyWorkingSet (or SetProcessWorkingSetSize) at your own risk. Perhaps only use it once after initial launch of your application. I have decided to disable the code and let the Garbage Collector manage memory usage on its own.