I have been working with storage spaces for a while now and there has been very little documentation that is clear on how to best tune it. Over time I have gathered a bunch of best practices which I have listed below. hopefully some of them might come in handy for you. Recently some new information form Microsoft was also released so its a good read.
PLEASE NOTE: This is a area that is changing allot depending on recommendations from Hardware Vendors and Microsoft so changes may apply, but I will try to keep this post updated.
First off read this new design guide and design calculator that was just released.
- Read the Design Consideration Guide found here.
- Use the Design Calculator found here.
Always configure your Storage pools via Powershell
- Update Drivers and Firmware to the latest supported version
- Install Latest Hotfixes (Markus has a great list here from 4th June 2015)
- Disable Trim (fsutil –set disabledeletenotify 1)
- We have found 4-6 Column count to work very well for most workloads (Dont forget to reduce with 1 disk for failures)
- Spread Virtual disk across SOFS Nodes
- Leave the Write Back Cache to 1GB for best experience at fail-overs.
- Max 80 Disks in Pool, Max 4 Pools per Cluster, Max 64 Virtual Disk/Storage Spaces per pool. (Max 240 Disk in Total)
- Keep Virtual Disks/Storage Spaces to no more then 10TB
- Use 64kb Interleave and Block Size for VM Workloads
- If you want to be able to loose a whole disk enclosure use minimum 3 enclosures and don’t forget to enable enclosure awareness.
- Reserve ~10-15% for SSD and ~15-20% for HDD of free space for faster rebuilds
- Choose RepairPolicy – Paralell is faster but can impact I/O and Sequential is slower with less I/O Impact
- Optimise Parallel Disk Rebuilds Technet Link
- Set RetireMissingPhysicalDiskPolicy to Always for Auto Repair (Auto Repair will kick in 5 minutes after first failed write)
- Most scenarios Include Tiering and sometimes you need to reschedule the Schedule Task that does the move of hot and cold data, so keep that in mind. (Microsoft doesn’t recommend running this task more than 4 times per day.)
- If you will use SharedVHDX you may need to apply this hotfix KB3025091 or configure your Pool with -LogicalSectorSizeDefault 512
- Set MPIO Default Settings to Load Balancing Set-MSDSMGlobalDefaultLoadBalancePolicy –policy LB HDD Should have LB and Round Robin for SSD (Markus has a nice script here.)
For the SMB Communication we have been using RDMA with Mellanox Connect X3 Pro Cards with good results. Dont forget to configure it for Multichannel.
Other Storage Spaces Resources
Storage Spaces FAQ
Dell Deployment Guide
Spaces-Based, Software-Defined Storage: Design and Configuration Best Practices
Architecting Software Defined Storage: Design Patterns from Real-World Deployments
Best Practices for Deploying Tiered Storage Spaces in Windows Server 2012 R2
Storage Spaces – Scale-out file server deep dive
Microsoft Virtual Academy
Storage Spaces MVA
Automatic SMB Scale-Out Rebalancing in Windows Server 2012 R2
Step-by-Step for Mirrored Storage Spaces Resiliency using PowerShell
File Server Tip: How to rebalance a Scale-Out File Server using a little PowerShell