SMB 3.0 is Microsoft’s data protocol that was introduced in Windows Server 2012 (WS2012). This data protocol evolved to SMB 3.02 in Windows Server 2012 R2 (WS2012 R2). One of the features of SMB 3.0 is SMB Multichannel, which can cause unexpected issues if left unmanaged. In this series, I will explain why you need to manage SMB Multichannel, and how you can place constraints on the feature to control different NICs it uses. In this post, I’ll start by looking at how SMB selects a NIC to use.
SMB Multichannel is one of the features that was added in SMB 3.0 that makes this old protocol ready for enterprise-scale and cloud-scale data flow. The role of SMB Multichannel is to allow Windows Server (and Windows client, too) to make the most of whatever network connections are available to push data as quickly as possible from a client to a server. SMB Multichannel works in two ways:
It is the latter issue that is commonly a problem. Some Windows features use SMB Multichannel, where we see unexpected use of networks. Alternatively, there are scenarios where people have implemented network designs, but they don’t get the results that they expected because SMB Multichannel went a different direction. I’ll explain why these things happen and how you can control them in the rest of this post.
There have been scenarios where people have installed unusual network designs in their Hyper-V hosts and they get SMB 3.0 travelling across NICs that they did not expect. The following diagram illustrates such an example. The Hyper-V host has been deployed using System Center Virtual Machine Manager (SCVMM) bare metal deployment. The consultant incorrectly thought that they had to deploy the host with a physical NIC for the management OS, and they use management OS virtual NICs for SMB 3.0 storage traffic. Now, the SMB 3.0 storage traffic is flowing via the physical management NIC (1 GbE) and not the dedicated SMB 3.0 storage virtual NICs (connected to 10 GbE NIC team).
Incorrectly designed SMB Multichannel storage. (Image: Aidan Finn)
There is a logical reason to this, and it is because of the process that SMB Multichannel uses to select NICs. NICs are chosen in the following order:
When SMB 3.0 looked at the available NICs the ordering was:
Note that Windows Server 2012 R2 does offer virtual RSS, but this feature is not available to management OS NICs. This is why the SMB 3.0 traffic will not select the provided SMB1 and SMB2 NICs.
Note: When building a scale-out file server, you also need to ensure that you configure the DNS settings (TCP settings) and enable client access (Failover Cluster Manager) for the desired storage NICs.
Windows Server uses SMB 3.0 for more than storage since WS2012 Failover Clustering has used SMB 3.0 for redirected IO. And those of you that are lucky enough to have 10 GbE or faster of bandwidth for Live Migration, then you can enable SMB Live Migration. Both of these features use SMB 3.0 the same way that Hyper-V on SMB 3.0 does. And SMB Multichannel selects NICs in the same way. So it is important to understand this process when you are designing Hyper-V host or storage networking.
In part 2 of this series, I will show you how to restrict SMB Multichannel to selected NICs to avoid spill-over into other networks.