How to Virtualize iSCSI Target Servers on Azure Stack HCI
Use virtual iSCSI servers as a target for other workloads like SQL and SharePoint Server
Last Update: Nov 19, 2024 | Published: Oct 07, 2024
Today I would like to dig a bit deeper into guest virtual machine (VM) storage virtualization on Azure Stack HCI. My case and scenario today will be around virtual iSCSI servers as a target for other workloads like SQL Server.
My cluster consists of three Azure Stack HCI 23H2 nodes cabled in a switched configuration. There are two SQL Servers also running on the cluster. As iSCSI targets, I will use Windows Server 2022 and 2025 with the Fileserver and iSCSI roles enabled.
In addition, we will have one physical domain controller for our Azure Stack HCI fabric, to start up and maintain cluster authentication in case of a cluster reboot. There will also be two more domain controllers running on the cluster for redundancy purposes.
Why do we still require classic storage?
Most modern applications support Server Message Block (SMB) or Network File Sharing (NFS) as storage target but you often have legacy software or features which requires older SAN protocols.
In Windows Server Hyper-V you can path-through physical Fiber Channel Adapters directly into VMs but it’s often a complex solution and it requires expensive hardware. In most environments, it makes sense to virtualize iSCSI storage servers as they can leverage virtual network adapters and the common IP protocol.
Virtual iSCSI Server as target systems
In my example, I will describe practices for Windows Server as an iSCSI target but you can use any other iSCSI software that Azure Stack HCI supports, like iSCSI tools for Linux, PetaSAN, and Open-iSCSI.
With the Windows Server Storage feature you also get benefits like compression, as well as storage tiering. Storage tiering is useful as it can be used on our Azure Stack HCI host system.
But to make use of storage tiering, you need to follow some basic guidance for the storage and iSCSI VM layout.
VM layout
Firstly, we want to have a cluster of at least two iSCSI target servers for load balancing and redundancy of our workload and environment. Both targets should make use of the preferred host or anti affinity rules in Azure Stack HCI, that prevents both iSCSI VMs and the workload VMs from ending up on the same Azure Stack HCI host system.
I prefer anti affinity over preferred host, as that prevents clustered or redundant VMs ending up on the same host but it does not prevent cluster load balancing. Especially with an uneven number of nodes within a regular N+1 cluster, it allows balanced resource usage, like shown in the picture below.
VM and host storage layout
When it comes to storage layout, you need to take three layers as well as certain features into consideration. The layers to consider are the physical layer and disk layout in your Azure Stack HCI hosts. If you run only one configuration, like an all nonvolatile memory express (NVMe) or all solid-state disk (SSD) layout, there is not much to consider as you can only create flat volumes without additional tiering options.
The complexity comes with a cache and capacity tier layout mixing NVMe, SSD, and hard disk (HDD). Those tiering options are only available if you ran a configuration where you have NVMe, HDD and SSD storage or configurations using four or more servers.
You don’t want to waste your high performance tier for storage workloads, which do not require a high number of input/output operations per second (IOPS).
This also applies to virtualized storage systems. From a physical layer perspective, you would create different storage volumes for your storage VMs.
With your storage VMs, you want to put low performance applications and usage on the Azure Stack HCI volume using for example HDDs. Storage volumes and performance on your storage virtual machines are often defined by the application servers.
SQL Server and SharePoint Server should have two or more storage targets connected: One for the transaction logs and database, which should be fast; and another for other server components, which are normally not so demanding.
When it comes to performance calculation in general, there are several tools that help you to size your servers and storage VMs.
Vendors, like Dell, have tools available but there are also general tools like the Azure Stack HCI sizing tool. For a more detailed and vendor independent calculation, there is a tool called ScopeSys from Acuutech. ScopeSys requires a license but it is more powerful than most other tools on the market. Check out our tutorial on using ScopeSys. There’s also a guide on how Azure FastTrack works and how to create a bill of materials (Bom).
VM network layout
With storage VMs running on a Azure Stack HCI host, only serving other VMs on the same host, there is not much you need to be concerned about. But it becomes complex when servicing VMs on another host or iSCSI initiators and clients outside of the cluster.
The first issue is the virtual network interface layout. If you want to use features like Multipath I/O, you will require at least two virtual network interfaces. In addition, I would always add another interface for management and backup traffic. That brings us to the following layout for the virtual machine.
It also makes sense to use SR-IOV for the storage interface of your VMs to increase performance.
Physical host network layout
From a physical host perspective, if you want to run virtual machines with a high network I/O requirement, it doesn’t make sense to use a fully converged configuration on two network interfaces.
When you run all workloads through the converged interfaces, you would easily hit performance bottlenecks if you do not switch to high-capacity interfaces like 40 GBE and more. So, I personally prefer a fully disaggregated deployment.
Most servers today offer at least four to six network interfaces. Most servers use two 25 GBE for storage, two 10 GBE interfaces for compute, and another two 1 or 10 GBE interfaces for management/backup or additional virtual switches.
For optimal performance and reliability, I recommend having additional 10 GBE interfaces instead of 1 GBE and dedicate those to your virtual iSCSI target systems.
Be aware that especially with Azure Stack HCI and Windows Server Hyper-V, you cannot mix different network adapters. So, every adapter must be from the same vendor, have the same firmware and driver version, and must be on the same kind of interface. It is not possible to mix LAN on Motherboard with PCIe adapters and it’s not possible to mix fibre optic with copper interfaces e.g. LC-LC with RJ45 Interfaces.
Which hosts are the best?
Normally, you can take any host from any supported vendor. But if you plan for further extension or want to add additional hardware later, e.g. GPUs or AI accelerator cards, you should choose a larger chassis and storage backplane, giving you expansion space for the future.
Here are some examples servers you could use:
More options can be found in the Azure Stack HCI Sizing tool.
Plan and know your workload
Even if most customers and environments want to reduce or completely cut their Storage Area Network (SAN) environments, there are often requirements from applications, which force us to use at least virtual SAN appliances like iSCSI targets.
Overall, everything comes down to planning and knowing your workload. There are always workloads you should not virtualize, like Oracle databases, as the licensing model could become expensive.
For other workloads, you should work with your application partner or expert, to plan and size a proper VM environment.
Having a multi tier storage and flexible network setup is important for future growth. Especially with a mixed workload, you want to have a flexible hardware setup.