Today I’m going to follow up my overview of Windows Server 2012 (WS2012) Hyper-V Live Migration with a look at the new features and improvements of Live Migration in Windows Server 2012 (WS2012 R2) Hyper-V and Hyper-V Server 2012 R2.
Live migration moves virtual machines from one location to another with no perceivable downtime to service delivery. It was first present in Hyper-V in Windows Server 2008 R2; there it could move the memory and process of a virtual machine from one host to another within the same Hyper-V cluster. The virtual machines’ storage did not move, instead, it stayed static on the shared cluster storage.
Recently Windows Server 2012 added signification improvements to Live Migration, including the following.
That covers the “how” of live migration but not the “why” of live migration. The purpose of this feature, just like with VMware vMotion, is to provide agility and flexibility. The infrastructure needs the ability to adapt to the changing demands of services that are being delivered and the requirements of the business – without enforcing downtime on the business.
Thanks to live migration we can:
Those are just some day-to-day examples of the many reasons we (or System Center on our behalf) would use live migration to move virtual machines around the data center. This all sounds fantastic. How could Microsoft top WS2012 live migration? Surely this would be impossible in such a short time frame? Well, Microsoft did surprise us when they announced the new live migration features in WS2012 R2.
Before we even talk about the coolest features of WS2012 R2 live migration we need to discuss how we get from WS2012 Hyper-V to WS2012 R2 Hyper-V. This topic, as with the rest of the discussion, includes the free Hyper-V Server. Roughly speaking, there are two generic ways to get your existing virtual machines on WS2012 Hyper-V to run on WS2012 R2 Hyper-V:
Migration offers the least amount of downtime during an upgrade, but there is still downtime. And that’s unfortunate for any service provider whether the customers are internal or external. It’s bad enough if a new version of Windows Server is released every three years, but imagine all those engineering hours at 3 a.m. on a Saturday if Microsoft was to release a new version of Windows Server just 12 months after the last one and continue to keep up that pace… oh wait – they are doing that now.
This is why cross-version Live Migration was added to WS2012 R2 Hyper-V. With this feature we can build a host with WS2012 R2 (this could be an existing WS2012 host that is drained and rebuilt) and perform a one-way-only from WS2012 Hyper-V to WS2012 R2 Hyper-V Live Migration of virtual machines to the new host.
This does give us the ability to do a zero-downtime-to-service upgrade of our hosts or clusters. Don’t forget that you’ll need to deploy updated integration components to virtual machines that will require a reboot, but at least this can be automated and scheduled using something like System Center Configuration Manager (see collection maintenance windows).
Cross-version live migration for zero-downtime cluster rebuilds
There are limits to cross-version live Migration:
Sadly, we cannot do an in-place upgrade of a cluster. Microsoft is aware of our request for this, and cross-version live migration will ease the pain somewhat, but many will choose to go with the CSV migration approach of the Cluster Migration Wizard because they have limited storage capacity.
One of the nicest improvements of WS2012 (not restricted to just live migration) was the ability to use the full capacity of 10 Gbps or faster networking. Hosts have increasing levels of capacity and virtual machines are getting bigger and bigger (up to 1 TB RAM in WS2012/R2 Hyper-V). It takes time to (a) copy and (b) synchronize the memory of those virtual machines between two hosts and that can introduce delays in planned or unplanned emergency maintenance. Adding 10 GbE or faster that can be used by live migration greatly reduces that time, maybe getting virtual machines off of a failing host before an interruption to service occurs.
Microsoft recognized that there was a need to optimize live migration beyond the algorithm improvements in WS2012. Even non-optimized 10GbE might not be enough for massive hosts. Many customers have investments in 1 GbE networking and have no plans to upgrade soon. This is why Microsoft has given us three types of live migration in WS2012 R2:
These options can be found in the host settings in Hyper-V Manager:
Configuring live migration method in WS2012 R2 Hyper-V.
The processor capacity of hosts is generally underutilized and Microsoft leverages this using compressed live migration. With this option enabled, live migration will use the spare processor capacity of the host to compress live migration on the source host and decompress it on the destination host. Hyper-V is very careful; it monitors the demands on the processor by higher priority tasks, such as virtual machines, and prioritizes their needs. So if there is no spare processor capacity, live migration traffic will not be compressed.
Live migration compression is the best option when you have 1 GbE networks. The improvements in live migration times are significant in the typical host where processor resources might be 25-33% utilized.
Server Message Block (SMB) is the protocol used by Microsoft for file services. WS2012 introduced a new version called SMB 3.0. This added two significant new features to this data transfer protocol:
WS2012 R2 Hyper-V can not only use SMB 3.0 networks for storing virtual machines on file servers (as with WS2012) but it also adds a new trick: WS2012 R2 Hyper-V can use these super-fast high-bandwidth networks to perform Live Migration. The recommendation from Microsoft is to use SMB-powered live migration when you have 10 Gbps or faster networking:
Live migration is the fastest option – assuming you have 10 GbE or faster networking – and for many virtual machines this brings the time required for each virtual machine down to the lowest possible theoretical time. There is a certain amount of time required to “build up” the virtual machine on the destination host, copy/synchronize the memory, and then “tear down” the virtual machine on the source host. Networking can improve the copy/transfer of the virtual machine, but it has no impact on the “build up” or “tear down.”
Most of the conversations about WS2012 R2 live migration have stopped by now. There is one other improvement in WS2012 R2 that affects live migration. NIC teaming has a new load balancing mode in WS2012 R2 called Dynamic. This type of traffic distribution uses flowlets to spread the inbound and outbound traffic of a single data transfer across all of the team members (physical NICs) of a NIC team. This means that a team of 1 GbE NICs with compressed live migration could spread their transfer across all of the team members, thus giving you the extra bandwidth effect that SMB Multichannel gives 10 GbE or faster NICs.
Note that NIC teaming and RDMA are incompatible.
In tests, we have seen the following: