Clustered Hyper-V Virtual Machine Prioritization

Windows Server 2012 (WS2012) gave us a new feature in Failover Clustering that allows us to prioritize our highly available (HA) virtual machines. When you first read of this feature, many articles often imply that it is used just to order the failover of virtual machines. However, clustered virtual machine prioritization allows us to control how the cluster will behave in several different scenarios. In this article, I will explain how you can configure priorities for HA virtual machines and discuss the implications of and use cases for this feature.

Configuring Highly Available Virtual Machine Priority

Virtual machine priority is a feature that is only available to HA virtual machines, and those are virtual machines that are running on a Hyper-V cluster, based on WS2012 Hyper-V or later. There are four possible priorities that you can assign to a virtual machine:

  • High (3000)
  • Medium (2000): The default setting
  • Low (1000)
  • No Auto Start (0)

As you might guess, high priority virtual machines are more important than medium and low priority virtual machines. The No Auto Start value is used typically for non-production virtual machines; these are machines that you do not want to start up after a cluster failover occurs so that you can prevent these virtual machines from consuming resources.
Setting a value for a virtual machine is pretty easy in Failover Cluster Manager. By default, every virtual machine is given a medium priority. You can select one or more virtual machines, right-click, click Change Startup Priority, and then select one of the four values.

Configuring the priority of highly available Hyper-V virtual machines in Failover Cluster Manager
Observing Hyper-V virtual machine priority. (Image: Aidan Finn)

Each setting has a numeric value shown above, where this value is stored with the cluster group that is used by Failover Clustering to manage the HA virtual machine. You can set a virtual machine with a low priority using the following example:

​(Get-ClusterGroup VM01).Priority=1000

In theory, it is possible to use interval values such as 1500. However, this is not supported and will lead to unpredictable results. In other words, only use the values that Microsoft provides: 3000 (High), 2000 (Medium), 1000 (Low), and 0 (No Auto Start).
You can quickly report the priorities of virtual machines using this snipped:

​Get-ClusterGroup | Where-Object {$_.GroupType -eq "VirtualMachine"} | Select-Object Name, Priority

Getting highly available Hyper-V virtual machine priority using PowerShell
Using PowerShell to query highly available virtual machine priorities. (Image: Aidan Finn)

As you can see, configuring this feature is easy, but what are the effects of configuring a virtual machine priority?

Modelling Application Dependencies

The most common usage of virtual machine priority is to model the dependencies of guest services. A simple example is tiered application:

  1. An application server depends on a SQL Server.
  2. SQL Server relies on a domain controller
  3. The domain controller must be running for the service to start

Using this dependency tree, the virtual machines are prioritized:

  • Domain controller: high
  • Database server: medium
  • Application server: low

Virtual Machine priority can be used to order the failover of highly available Hyper-V virtual machines
Modelling application dependencies. (Image: Aidan Finn)

Imaging that all three virtual machines are running on Host1, where the host suffers a sudden failure. This is when the cluster heartbeat and failover process kicks in. Other nodes in the cluster detect that Host1 is offline when it fails to heartbeat. The other nodes will initiate a failover process and the virtual machines are failed over and started in this order:

  1. Domain controller
  2. Database server
  3. Application server

Note that this is a simple process. It does not verify that guest services are functional before starting the dependent virtual machines. And we are also restricted to just three tiers — high, medium, and low — for our multi-tiered applications.
I think that last sentence gives away what Microsoft’s developers were thinking when they created the priority feature. This feature is for more than just modeling applications. In fact, I think this modelling capability was low on the ladder when the feature was designed.

Draining a Host

We can drain a clustered host of virtual machines in two ways:

  1. We use the pause action before carrying out maintenance, either manually or automatically (Cluster Aware Updating)
  2. We shut down a WS2012 R2 host that still is running virtual machines

A pause action will allow you to choose the Drain Roles option. This will migrate virtual machines to the most suitable host (based on RAM availability) in the cluster – note that System Center Virtual Machine Manager (SCVMM) uses a more informed Intelligent Placement feature.
What you should see is that high-priority virtual machines will be moved via Live Migration before medium or low-priority virtual machines. However, there is an exception to this rule, which you will read about below.
What happens to low-priority virtual machines by default depends on your version of Hyper-V. In WS2012, low-priority virtual machines are move using Quick Migration. Quick Migration was the precursor to Live Migration, where virtual machines are put into a saved state, failed over to the destination host, and restored from the saved state. It is quick, but it features downtime. What was Microsoft thinking?
The logic for this behavior in WS2012 was that most customers still used 1 GbE networking for Live Migration. The customer wanted virtual machines off of the host as quickly as possible. High and medium priority virtual machines would migrate with no perceivable downtime on the Live Migration network, and low-priority virtual machines would migrate with a small amount of downtime via the storage network. This process leverages two sets of networks and gets the host offline as quickly as possible – and low-priority virtual machines ‘must not be that important.’
Microsoft received negative feedback for this default behavior on WS2012 Hyper-V and changed it in WS2012 R2 so that all priorities would use Live Migration – we also get more efficient compressed and SMB-based Live Migration in WS2012 R2.
You can control the behavior of low priority virtual machines using a cluster value called MoveTypeThreshold, which you can see using this PowerShell snippet:

​Get-ClusterResourceType "Virtual Machine" | Get-ClusterParameter MoveTypeThreshold

On WS2012 R2 this is set to 1000, meaning that virtual machines of 1000 or higher will use Live Migration. On WS2012, it is set to 2000, meaning that virtual machines of 2000 or higher will use Live Migration. We can manipulate this value. For example, if you want to use Live Migration for low-priority virtual machines on WS2012 hosts then run:

​Get-ClusterResourceType "Virtual Machine" | Set-ClusterParameter @{MoveTypeThreshold=2000}

Note that low-priority virtual machines will move via Quick Migration in parallel to high and low priority virtual machines moving via Live Migration. And machines marked not to auto start will also move via Quick Migration, but they will not complete the startup process on the destination host.

Memory Contention

With Dynamic Memory we should be able to squeeze lots of virtual machines onto fewer hosts. But there are times when we run out of resources and hard decisions need to be made. Failover Clustering will use priority to automatically make those decisions.
If a higher priority virtual machine requires a large amount of memory that is not available on the destination host, lower-priority virtual machines are automatically put into a saved state; this happens before the higher priority memory hungry virtual machine can fail.
So as you can see, configuring priority for your HA VMs does more than just model application dependencies. It can also be used to control the speed of migration during planned host maintenance, and it can be used to control the sharing of RAM between virtual machines. So keep these implications in mind when you are planning virtual machine priority.