Framework version

If you encounter Memory Exception problems, the first thing you should do is to make sure that you have Service Pack 1 for .NET Framework 1.1 installed. Memory Exceptions will occur more frequently if you have .NET FW 1.1 without the SP and they will also occur during much lower memory loads.

To make sure that you have SP1 installed use the registry editor to check the value of the key HKEY_LOCAL_MACHINE\Software\Microsoft\NET Framework Setup\NDP\v1.1.4322\SP. The value corresponds to the latest service pack installed.

EPiServer 4.51 Hotfix 4 and later

Starting with 4.51 hotfix 4, there is a feature in EPiServer that may conserve very large amounts of memory on high content-site. If you site is regularily using 1Gb or more of memory (after following the other advice here, allowing such amounts to be used safely), you may benefit from this feature. In brief, it allows for selective compression of LongString and derived properties when stored in the memory cache, this can yield benefits up to a factor of 10 or more in reduced memory requirements. Please see the documentation for hotfix 4 for further details.

Microsoft background information

The following Microsoft article is a very good reference when you are experiencing performance problems or when you are experiencing OutOfMemoryExceptions: http://msdn2.microsoft.com/en-us/library/ms972959

Especially important is the information about Private Bytes, it can be found under the the "Process Counters" heading close to the end of the article. There is quite a lot of information to be found there but this is the really important bit:

"To summarize, set the memory limit to the smaller of 60% of physical RAM or 800 MB. Since v1.1 supports 3 GB virtual address space, if you add /3GB to boot.ini, you can safely use 1,800 MB instead of 800 MB as an upper bound."

Please note that IIS 5 and IIS 6 use different ways of specifying "memory limit".

IIS 5: The memoryLimit attribute of the processModel tag in machine.config, expressed in percent of physical memory.
IIS 6: The "maximium used memory (in megabytes)" under the application pool properties, expressed as indicated in megabytes. Note: IIS 6 ignores the machine.config memoryLimit-attribute setting.

The "memory limit" is compared against the process counter "Process\Private Bytes" and is used for two things in this context:

• It controls when the IIS worker process is "recycled", i.e. the old one is stopped and a new one is started - essentially an automatic web site restart.
• It controls when System.Web.Caching.Cache determines that it is time to purge things from the cache - to avoid process recycling.

"Process\Private Bytes" is a measure of the total amount of memory allocated by the process for it's own use, that cannot be shared with other processes. This is not the same as the amount of virtual memory allocated, in fact it will always be smaller since a process may have virtual memory that is not private, but no private memory can exist that is not also part of the virtual memory space.

According to Microsoft, in the article referenced above, OutOfMemoryException may occur when "Process\Virtual Bytes" is within 600Mb of the virtual adress limit (which in current servers is either 2Gb or 3Gb if the /3GB flag is used in boot.ini). It is unclear why there need be a 600Mb margin, but the point is that Microsoft states that we need that margin.

Microsoft also states that "tests have shown" that "Process\Virtual Bytes" is seldom more than 600Mb  larger than "Process\Private Bytes".

Added together this means that: IF "Process\Private Bytes" + 600Mb + 600Mb > the virutal adress limit THEN the risk of OutOfMemoryException increases.

The virtual adress limit is, as previously mentioned, either 2Gb or 3Gb. Since the framework will try to keep the "Process\Private Bytes" below the "memory limit", this means that the "memory limit" should not be set larger than 800Mb or 1,800 Mb depending on the /3Gb switch setting.

The /3GB switch determines the split between process and kernel memory in the 4GB virtual adress space provided by 32-bit Windows. It has nothing to do with physical memory. Since an ASP.NET application after deducting safety margins and memory overhread for the worker process has about 600Mb memory available, increasing the available virtual adress space by 1GB almost triples the amount of memory available for the ASP.NET process.

The amount of memory available is per application pool, so you can have larger processes running, if you run them in separate application pools. Once the amount of active memory used (working set) in total for all application pools, the system and other processes exceeds the physical memory size, it will keep on running but slower since you will now experience heavy paging.

Tuning the memory settings is thus a question of being aware of how your applications use memory, but normally you should enable the /3GB switch regardless of physical memory size, and set each large ASP.NET application in it's own application pool, and there set the "memory limit" to 1,800 Mb. This will minimize the risk of OutOfMemory exceptions - at the possible expense of performance of course.

Since a lot of memory is allocated by the cache, i.e. "non-vital" memory, you can thus increase performance in some cases by reducing the amount of memory available to each application pool so as not to exahaust physical memory. Then again - paging may well be more efficient than re-retrieving the data from the source.

Performance tuning is a very difficult sport, so start with the recommended settings to minimize the risk of OutOfMemory exceptions - it's almost always better to run, albeit slowly, than crash.

 In summary:

• The memoryLimit attribute for IIS 5 is set in percent of total physical memory so you have to do some calculations when you want to set a fixed number of MB.
• An individual application pool should have it's "memory limit" set to either 800Mb or 1,800Mb depending on the /3Gb-switch setting (or possibly lower in some cases to optimize performance).
• The /3GB switch in boot.ini is valid regardless of the amount of physical memory.
• Without the /3GB switch, you may even get OutOfMemory exceptions while you still have lots of physical memory free!
• IIS 6 ignores the machine.config settings for memoryLimit - use the IIS 6 admin ui from the application pools node to set the value there ("Maximum used memory (in megabytes)").