Choosing Hyper-V Storage: Virtual Hard Disks
We recently looked at the ways that Hyper-V virtual machines could use physical disks, and we’ve discussed the benefits and flaws of these kinds of storage. This week we are going to look at the modern, business friendly, cloud-enabled, virtual hard disk alternative.
Flaws of Physical Disks
The principal flaw is derived from the fact that a physical disk, such as a pass-through disk, is not abstracted from the physical LUN that enables it. The pass-through disk requires exclusive use of the LUN. A physical storage LUN cannot be moved using Live Storage Migration or Shared-Nothing Live Migration. We cannot backup a virtual machine from the host using a Hyper-V-ready VSS snapshot; instead, we need to install expensive (both in effort and complexity) backup agents into the guest OS to back up the physical disks.
Granted, these are all technical reasons – which the board of directors and shareholders don’t care about. However, they do care about the following factors:
- Being more flexible and responsive to change: Whether deployed by IT or through a cloud (self-service), services that run on virtual machines are slow to deploy if a layer of physical infrastructure configuration (by humans) is injected into the process. This is a necessary step for physical storage deployment to virtual machines.
- Reducing costs: There is more to a business’s cost calculations than the cost of purchase. How much does something cost to deploy or configure? What is the cost of change? What are the costs of lost opportunities while waiting for IT to deploy complex configurations?
Whether you like it or not, businesses are moving towards cloud computing, be it public (multi-tenant service provider), private (infrastructure that is exclusive to the business), or hybrid (services deployed to public and private clouds). A key trait of this is self service. For example, a department is deploying a new service to the business and they do this using a console that IT provides. The service is deployed within a matter of minutes without any “delays” being “caused” by the IT (in the perspective of the business). Using physical disks will prevent this. A solution that abstracts storage is required.
Virtual Hard Disks
A virtual hard disk is a file that simulates a hard disk. From the outside, the file looks like any other file in the file system. When attached to a virtual machine’s virtual storage controllers, the virtual machine will mount the file and use it just like a physical disk: volumes are created, formatted with a file system (such as NTFS), and files/folders are created, managed, and deleted – just as if the disk was physical. From the outside, we see almost nothing of this going on inside of the virtual hard disk. Just like physical disks, a virtual hard disk has a size (or a maximum size) that can be expanded as required.
Virtual hard disks are one of the two data center containers that Microsoft uses. The other is the Windows Imaging (WIM) file format. WIM files are used in operating system imaging. You’ll find an Install.WIM file in every Windows ISO/DVD since the release of Windows Vista and Windows Server 2008. This is the same format that is used by ImageX, Windows Deployment Services, Microsoft Deployment Toolkit, and System Center – Configuration Manager.
Virtual hard disks (often shortened to VHD, which can be confusing, as you’ll soon see) are the other strategic container. Virtual hard disks are used for virtual machines, but they’re also are used for things like booting servers from a file instead of physical volumes (easy to deploy), Windows Server Backup, and the Windows Server iSCSI Target. Virtual hard disks offer benefits such as:
- It’s easier to move/copy a file instead of a volume.
- You can mount a virtual hard disk to scan/modify the contents.
- You can modify/patch an operating system in an offline virtual hard disk.
- Server deployment is much easier if you just copy a virtual hard disk that contains a generalized operating system , which makes self-service virtual machine a reality (using an IT-managed service and console).
There are three kinds of virtual hard disk in Hyper-V: Dynamically Expanding, Fixed Size, and Differencing. Now let’s take a closer look at each one.
Dynamically Expanding Virtual Hard Disks
Often referred to as dynamic, this is the default type of file when you create a virtual hard disk. It can be thought as a thinly provisioned virtual hard disk. This file starts out very small with a preset maximum size. The file grows in chunks as data is stored in the file, and it will stop growing when it reaches the maximum size.
The major benefit of this type is that the dynamic disk takes up only slightly more space that the data contained within. Roughly speaking:
- a virtual hard disk that has a maximum size of 200 GB
- and contains 10 GB of files
- is just over 10 GB in size
- that means the business saved nearly 190 GB of physical storage space
What are the downsides of dynamic virtual hard disks? One might think the gradual growth of the files would be an issue. Actually, real-world experience doesn’t show it to be an issue – Microsoft has worked to improve growth performance with this type with the releases of Windows Server 2008 R2 and Windows Server 2012. The real issue is read performance.
Imagine 20 small dynamic virtual hard disks on the same physical volume, each separately gradually growing a block at a time. Those blocks get fragments across the volume. That means that sequential reads suffer because the underlying physical disk spindles spend time bouncing around from block to block within the volume.
Fixed Virtual Hard Disks
The file of a fixed virtual hard disk is created at the maximum size of the volume. Using the same example as before:
- a virtual hard disk that has a maximum size of 200 GB
- and contains 10 GB of files
- is just over 200 GB in size
- that means the business nearly 190 GB of overused physical storage space
In the eyes of some IT pros, that 190 GB is wasted, while others hate that creating a fixed virtual hard disk is a slow process. Windows will zero out the contents of the new virtual hard disk as it is being created. This is a security measure to ensure that any underlying data in the physical storage layer cannot be accessed by a disk-scanning tool from within a virtual machine – a matter of great importance to a security-aware or data protection-conscious business! That delay-to-creation time can be eliminated by using Offloaded Data Transfer (ODX), a feature that is in some modern SANs.
Fixed virtual hard disks do not have dynamic growth, so they are not as prone to fragmentation as dynamic virtual hard disks. However, it can still happen if you manually expand a fixed virtual hard disk, or the virtual hard disk is created in a pretty full volume. Fixed virtual hard disks perform nearly at the same speed as the physical LUN that is storing them. This counters the argument for using pass-through disks for performance.
Differential Virtual Hard Disk
The differential virtual hard disk is intended for virtual machines that will have a short life, such as pooled virtual desktop infrastructure (VDI) virtual machines, or test labs. The concept is that a virtual hard disk with a generalized operating system is stored in a shared location. Lots of new virtual machines can be rapidly provisioned with differential virtual hard disks. Each differential virtual hard disk is a child that links to the shared virtual hard disk (the parent). The differential hard disks start out as small files and grow as new data is stored in them. The new virtual machines use the generalized OS from the parent disk to boot and specialize the OS install. New data is stored in the small (and growing) child disk and nothing is stored in the non-modified parent disk. Over time, the child disk grows.
While this is a superb way to build VDI or lab virtual machines that might live for a few hours, it is an absolutely horrendous idea to use this type of virtual hard disk for normal long-lived virtual machines. Differential disks come with a growing performance penalty and have unpredictable storage utilization patterns that have been known to cause outages if they are left operational for too long.
Do not use differential virtual hard disks for production server virtual machines!
Virtual Hard Disk Formats
There are two formats of virtual hard disk in Windows Server 2012 (Hyper-V) and Windows 8 (Client Hyper-V):
Now you can see where the acronym of VHD for virtual hard disk can become confusing!
The VHD format has been around since before Microsoft entered the virtualisation market. It came with Microsoft’s acquisition of a company called Connectix. VHD was designed to meet the challenges of the computer room over a decade ago. Back then everyone considered a 2040 GB volume (the maximum size of a VHD file) to be huge.
VHD format files were the only option in all previous versions of Microsoft’s virtualization products. However, Microsoft introduced a new format for a new decade with the release of Windows Server 2012 and Windows 8.
VHDX files are the new default format of virtual hard disks. They are not supported on legacy virtualization products, but you can convert a VHD file into a VHDX file and vice versa.
The biggest feature of VHDX files is that they scale out to 64 TB (also the maximum supported size of a VSS snapshot). This counters the reason for using pass-through disks for scalability in almost all scenarios but the most extreme (volumes larger than 64 TB).
Other benefits include:
- Logging updates to the VHDX metadata structures to defend against data corruption during power failures.
- Improved performance of the virtual hard disk on physical disks with physical sector sizes of 4 K.
- Support for VHDX files with 4 K logical sector sizes for improved performance with supporting guest operating systems and applications.
- Larger block sizes for dynamic and differencing disks.
- The ability to store metadata about the contents of the VHDX file.
- Support for TRIM to reclaim redundant physical storage utilization – assuming the VHDX is attached to a virtual SCSI controller and you have TRIM-ready hardware.
VHDX is the virtual hard disk format of today and the future, and it should be used for all deployments. There is a divide in the Hyper-V community over the best type of virtual hard disk to use for production servers. Some will only use Dynamically Expanding virtual hard disks. Others prefer Fixed Size virtual hard disks, but will use the dynamic alternative in a lab when disk space is restricted. The choice between the two is part-technical (performance) and part business (cost of storage).