Last Update: Sep 04, 2024 | Published: May 13, 2015
Microsoft started discussing a new feature of Windows Server 2016 called Storage Spaces Direct at Microsoft Ignite 2015. I’ll explain this new technology in detail, along steps on how you will be able to use it when Windows Server 2016 is released.
Windows Server 2012 and Windows Server 2012 R2 introduced us to an improved new form of Software-Defined Storage from Microsoft called a Scale-Out File Server (SOFS). Typically used with Hyper-V, the SOFS is a tier of storage, separated from the compute tier (Hyper-V). SOFS is typically made up of:
When implemented correctly, a SOFS is a fantastic way to deploy small and affordable clustered storage, and it’s also a great way to deploy huge amounts of Hyper-V storage for public or private cloud installations. Some experiences, however, were not so good:
Microsoft has invested a lot of effort to develop a new way to deliver storage, that is more scalable, easier to deploy, and offers more disk options. This has resulted in a new architecture called Storage Spaces Direct (S2D).
We are probably a year away from the release of Windows Server 2016 so a lot of what is written in this article is subject to change. However, the concept of S2D has been well explained by Microsoft. Instead of having a SOFS that is comprised of a cluster servers and a SAS network, S2D is a cluster that’s made of up servers with direct-attached storage (DAS). The supported disks include SAS HDDs and SSDs of the past, but S2D also supports:
The conceptual architecture of S2D looks like this:
Microsoft originally envisioned S2D as being a new way to deploy scalable and cost-effective storage for private clouds. Stated differently, the intention is to use S2D as an SMB 3.0 connected storage tier (SOFS) for Hyper-V hosts and clusters. File shares are created on the S2Ds virtual disks and the Hyper-V hosts store their virtual machine files on these shares, which means that the extents making up the files are actually stored in a fault tolerant manner across the servers of the S2D cluster.
You might have read an article that I wrote late last year where I discussed Microsoft’s view on hyper-convergence, as it was publicly stated at TechEd Europe 2014. Microsoft believed that hyper-convergence limited scalability. However, Microsoft has realized that they can use software (S2D) to create a hyper-converged solutions for small to mid-sized businesses. In this design:
There is no SMB loopback in this deployment model. Virtual machine files are stored on the CSVs that each host is mounting. Under the covers, the extents of the files are scattered all over the cluster members, and a virtual machine could be running on one node while CSV ownership is on another, where fast and low latency networking is required.
There are several layers of fault tolerance in S2D:
Note that a lower priority is used for recovery actions so that the performance foreground tasks, such as running virtual machines, is not impacted.
SMB Direct (RDMA) is required to network the nodes of a S2D cluster. RDMA provides:
One might question the cost of something like 40 Gbps ROCE v2, but this cost will be more than absorbed by the low cost of the storage system that is using SATA disks. This form of networking also allows hosts to make the most of SSD or NVMe disks.
Storage performance is enabled with tiered storage. A heat map tracks the popularity of data and moves hot data to the NVMe/SSD tier and colder data to the HDD tier. This offers the affordable scalability of SATA HDDs at the lower end and the speed of NVMe at the higher end, thus targeting spending in a very granular way. A write-back cache is enabled in tiered virtual disks to absorb spikes in write activity.
WS2016 ReFS also offers improves performance:
It is easy to add capacity to an S2D cluster. Simply add a new server with DAS storage to the cluster and rebalance the extents across each node, and note that this is a low priority IO task. You can also remove a server from an S2D cluster.
A few companies have invested in Storage Spaces with shared JBODs, and those that picked their hardware wisely realized great savings with awesome performance. Microsoft continues to invest in that architecture but S2D in combination with SMB 3.0 networking offers a new way to deploy interesting designs, both for large scale environments and small to mid-sized organizations.