Getting the most out of Hyper-V Dynamic Memory

Microsoft introduced Dynamic Memory in Windows Server 2008 R2 (W2008 R2) Service Pack 1 (SP1) to change how memory is assigned to virtual machines. Instead of assigning all of a virtual machine’s potential RAM to the virtual machine, you could opt to assign only slightly more than is required by the VM’s workload (pressure) to perform its tasks. Additional memory could be transparently added and taken away as pressure rose and subsided. This could increase the density of virtual machines you placed on your hosts (reducing CAPEX and OPEX for your hosts), but there are some important questions: Should you enable Dynamic Memory for all virtual machines? How should you configure Dynamic Memory? Should it be one-size-fits-all or should some workloads have a specific configuration?

Should You Enable Dynamic Memory for all Virtual Machines?

The answer to this one is quick: No, you should not enable Dynamic Memory for all virtual machines. Look back at how Dynamic Memory works. The guest OS of the virtual machine only ever sees the high point of RAM since it last booted up. So if a VM boots up with minimum RAM of 512 MB, it might raise to 768 MB. Now if you try to install something like SQL Server, the installer will block the installation because it cannot see more than 1 GB of RAM.

high point of Dynamic Memory assignment
In this example, not enough RAM was previously assigned to install SQL Server

Some products explicitly do not support Dynamic Memory enabled in the virtual machine in which it will be installed. For example, the Exchange Mailbox role does not support Dynamic Memory. If a virtual machine boots up with 2048 MB of Startup RAM and the Exchange Mailbox service starts, the service will take what memory it can and never look for more. It wouldn’t matter that the virtual machine could avail of up to 1 TB of Maximum RAM; Exchange will never attempt to get any of it. In this case you must not use Dynamic Memory, and you have to live with correctly tuning your RAM.

The support statement for Dynamic Memory changes based on product, version, and even edition. Older versions of SQL Server are good examples. SQL Server 2008 R2 Express did not support Dynamic Memory. SQL Server 2008 R2 Standard supported Dynamic Memory but could not use it. SQL Server 2008 R2 Enterprise could use Dynamic Memory, but there are very specific guidelines from Microsoft on how to tune the Dynamic Memory and SQL Server settings of a SQL Server virtual machine.
However, all the roles and features of Windows Server, such as Active Directory and IIS, do support Dynamic Memory. Always check the support statements for the product that you are installing to see whether Dynamic Memory is supported, and if so, what configurations are recommended or required. Note that this applies to Microsoft and non-Microsoft software.

Strategies for Dynamic Memory

There are a number of approaches that you can use for Dynamic Memory, and I will discuss some of them here.

Enable It No Matter What

This is the “please fire me” approach to IT. With this method, you enable Dynamic Memory no matter what. All joking aside, I can envisage some hosting companies using this as their default method in their public cloud and will offer a higher cost alternative if you want static RAM (no Dynamic Memory) for select virtual machines.
The benefit to the hoster is that they squeeze more virtual machines into their hardware; in my experience in the hosting business, there is a lot of wasted memory. Margins are tight, so every reused MB of RAM is the difference between profit and loss.

Carefully Tuned Dynamic Memory

With this approach, an engineer will calculate the required memory to start the OS and services and the maximum RAM that the workload will realistically require. Product guidance will also be used to configure weight and buffer settings. Some virtual machines will be deployed with static RAM.
The benefit here is that you get more virtual machines onto your hosts without sacrificing performance or support. The cost is that there is a lot of tuning, which is not appropriate in a self-service cloud.

Maximum Memory

This is an approach that you could use in a cloud, but there is a performance risk to it. You can configure virtual machines to have a Startup RAM value that is the same as the Maximum RAM. For example, a virtual machine will boot with 8192 MB RAM. The minimum RAM value might be 32 MB, so the machine might balloon down to 768 MB when idle. That saves over 7 GB of RAM on the host… eventually.
What if 50 virtual machines all try to boot up immediately in this way to their maximum RAM? This is where Hyper-V Smart Paging has to kick in. A paging file resides someone on host/cluster storage (with the virtual machine files by default) for each virtual machine. The virtual machine will page to the host storage to start up if there is insufficient host RAM to meet the needs of the virtual machine’s Startup RAM setting. Eventually the integration components will start up in the guest OS, and Dynamic Memory will balloon down. After a while, the virtual machine should close the Smart Paging file and rely on physical RAM only.
Note that using Dynamic Optimization (like vSphere DRS) in System Center Virtual Machine Manager (SCVMM) with a Hyper-V cluster would have prevented Smart Paging by load balancing virtual machines via live migration.
The maximum memory approach is a nice one for clouds. The helpdesk won’t get calls from irate customers who paid for 8 GB RAM but only see 768 MB when they log in. They’ll see the full 8 GB because the Start RAM setting is equal to the Maximum RAM setting. This will help with installer prerequisite checkers too. The risk is that in a rare scenario you might see Smart Paging kick in and reduce performance while virtual machines are booting up.

Elastic Memory

A trait of a cloud is elasticity – you can consume what you need. This means you take/pay more when you need it, and take/pay less when you need less. This is the approach that Dynamic Memory uses by default. If you enable Dynamic Memory while creating a virtual machine in the Hyper-V Manager wizard, you only set the Startup RAM. By default, Maximum RAM will be 1 TB, even if your host has only 32 GB RAM (Hyper-V never can assign more than the host has available). This might sound like an easy approach, but it leaves the host susceptible to bad programmers. Memory leaks within a virtual machine could eat up RAM on the host, all the way up to the limits of the host or the limits of the virtual machine (whichever is lower). That could affect other virtual machines and could cause a customer relationship nightmare if you have a pay-as-you-go billing system.
Remember: There are way more bad developers out there than there are good ones.

Balanced Approach

With this method you will have different templates in your library for different kinds of virtual machines. For example, I would have a hardware profile in SCVMM for basic VMs (with 512 Startup RAM), medium VMs (1024 Startup RAM), and non-enabled VMs (no Dynamic Memory).
There is a little setup work and the self-service user needs to select an appropriate image/template from the library/gallery for their workload. But this allows you to use Dynamic Memory in a balanced approach.

No Dynamic Memory

With this method you never enable Dynamic Memory because you have no idea what your tenants are going to install into your public/private cloud. You do not want the image of your service to be tarnished by tenants committing self harm, and you do not want your helpdesk to be inundated with calls. The decision is based on the benefits of Dynamic Memory being outweighed by the costs. Therefore, you never enable Dynamic Memory; the costs of your infrastructure will be higher, and therefore the costs to your tenants will be higher too, making your service less competitive.