Deep Dive: Storage Spaces Direct (S2D) 内部机制

Topic: Storage Spaces Direct (S2D) 存储架构与内部实现原理
Category: Storage
Level: 高级 / Expert
Last Updated: 2026-03-17


1. 概述 (Overview)

Storage Spaces Direct (S2D) 是 Windows Server 2016 引入的软件定义存储 (Software-Defined Storage) 解决方案,彻底改变了 Windows 集群存储的架构模式。传统的 Windows 故障转移集群依赖于共享 SAS 存储或 SAN 设备,而 S2D 则仅使用每个节点的本地磁盘,通过高速网络连接构建分布式存储池。

核心理念:将传统的”共享存储”模式转变为”分布式本地存储”模式。如果将传统 SAN 比作”多户家庭共享一个大车库”,那么 S2D 就是”每户都有自己的车库,通过气动管道 (RDMA) 连接实现资源共享”。这种架构消除了对昂贵共享存储硬件的依赖,同时提供更高的扩展性和性能。

S2D 的核心价值在于简化存储架构、降低成本,并通过软件定义的方式实现企业级存储的高可用性、容错性和性能优化。它不需要传统的 SCSI-3 Persistent Reservation (PR),因为所有的协调都通过集群服务和 Storage Bus Layer 完成。

2. Shared SAS vs S2D 架构对比 (Architecture Comparison)

传统 Shared SAS 架构

在 Windows Server 2012 R2 及之前的版本中,故障转移集群采用共享存储架构:

[节点1] ──┬── SAS/FC ──┬── [共享存储柜]
[节点2] ──┘            ├── RAID 控制器
[节点3]                ├── 磁盘阵列
[节点N]                └── SCSI-3 PR 锁定

特点

  • 每个节点通过 SAS/FC 连接到共同的存储设备
  • 需要专用存储网络(SAN)
  • 依赖 SCSI-3 Persistent Reservation 防止数据损坏
  • 存储扩展受限于存储控制器和机架空间
  • 单点故障风险(存储控制器)

S2D 分布式架构

S2D 彻底改变了这一模式:

[节点1-本地存储] ──┐
[节点2-本地存储] ──┼── 10GbE+ RDMA 网络 ──┐
[节点3-本地存储] ──┤                      ├── 软件定义存储池
[节点N-本地存储] ──┘                      └── 分布式镜像/奇偶校验

特点

  • 每个节点仅使用本地连接的磁盘
  • 通过 10GbE+ 以太网连接(推荐 RDMA)
  • 不需要共享存储硬件
  • 线性扩展:增加节点同时增加计算和存储
  • 无单点故障(分布式架构)
  • 不需要 SCSI-3 PR

3. Storage Bus Layer (SBL): S2D 的心脏 (Core Component)

Storage Bus Layer (SBL) 是 S2D 的核心组件,它创建了一个”虚拟的共享 SAS 总线”,让分布式的本地存储看起来像传统的共享存储。

SBL 组件架构

本地节点                    远程节点
┌─────────────┐           ┌─────────────┐
│ Spaceport   │           │ Spaceport   │
├─────────────┤           ├─────────────┤
│ ClusPort.sys│ ←──RDMA──→ │ClusBflt.sys │
│ (Initiator) │           │ (Target)    │
├─────────────┤           ├─────────────┤
│   本地磁盘  │           │   本地磁盘  │
└─────────────┘           └─────────────┘

ClusPort.sys (SBL Initiator)

功能:负责向其他集群节点发起存储请求

  • 作为 SCSI Miniport 驱动程序运行
  • 将本地的存储 I/O 请求转发到远程节点
  • 通过 SMB Direct (RDMA) 传输块设备 I/O
  • 维护到其他节点的连接池

ClusBflt.sys (SBL Target)

功能:响应来自其他节点的存储请求

  • 作为 Storage Bus Filter 驱动程序运行
  • 接收并处理远程节点的 I/O 请求
  • 实现 SBL Cache(读写缓存机制)
  • 管理本地磁盘的访问控制

隐藏的 SMB 共享

SBL 在后台创建隐藏的 SMB 共享:

\\Server\BlockStorage$\Device\1
\\Server\BlockStorage$\Device\2

特点

  • 专门用于块设备访问,不是文件共享
  • 通过 SMB3 协议传输 SCSI 命令
  • 支持 SMB Direct (RDMA) 优化
  • 对用户和应用程序完全透明

Block-over-SMB3 协议

SBL 实现了”Block over SMB3”协议,这是一个创新的设计:

  • SCSI 命令封装在 SMB3 消息中
  • 数据传输通过 SMB Direct (RDMA) 优化
  • 保持 SCSI 语义,确保与上层驱动兼容
  • 类比:在以太网管道中传输 SCSI 总线信号

4. 完整 S2D 存储栈 (Complete Storage Stack)

S2D 的存储栈比传统存储更加复杂,涉及两次通过分区管理器和磁盘驱动的过程:

物理磁盘路径 (Physical Disk Path)

应用程序
    ↓
ReFS/NTFS
    ↓
CSV (Cluster Shared Volume)
    ↓
Volume Manager
    ↓
Partmgr.sys (第二次)
    ↓
Disk.sys (第二次)
    ↓
Spaceport.sys ← 虚拟磁盘创建
    ↓
ClusBflt.sys (SBL Target)
    ↓
ClusPort.sys (SBL Initiator)
    ↓
Partmgr.sys (第一次)
    ↓
Disk.sys (第一次)
    ↓
Storport.sys
    ↓
Miniport Driver (如 stornvme.sys)
    ↓
物理磁盘硬件

双重通过的原因

第一次通过:处理物理磁盘

  • 物理磁盘被 Storport → Miniport → Disk.sys → Partmgr 处理
  • SBL 层获取原始磁盘访问权限
  • 创建分布式存储池

第二次通过:处理虚拟磁盘

  • Spaceport 创建的虚拟磁盘重新进入存储栈
  • 再次通过 Disk.sys → Partmgr → Volume Manager
  • 最终被 CSV 和文件系统使用

这种设计确保了 S2D 虚拟磁盘与传统磁盘在上层的完全兼容性。

5. Spaceport.sys: 存储空间管理器 (Storage Spaces Manager)

Spaceport.sys 是 Storage Spaces 的核心驱动程序,在 S2D 中承担关键功能:

主要职责

  1. 磁盘声明 (Disk Claiming)
    • 识别可用于存储池的磁盘
    • 排除系统磁盘、小容量磁盘等
    • 管理磁盘的生命周期
  2. 存储池管理
    • 创建和维护分布式存储池
    • 跨节点的磁盘聚合
    • 池的健康监控和维护
  3. 虚拟磁盘创建
    • 根据容错策略创建虚拟磁盘
    • 支持镜像、奇偶校验等不同的弹性类型
    • 动态分配和精简预配
  4. 容错处理
    • 检测磁盘和节点故障
    • 自动数据重构和修复
    • 维护数据的多副本

SpaceDump.sys

这是 Spaceport 的调试伴侣驱动:

  • 在系统崩溃 (BSOD) 时收集存储状态信息
  • 生成详细的存储池诊断数据
  • 帮助 Microsoft 支持团队分析存储相关问题

6. S2D 缓存机制 (Caching Architecture)

S2D 的缓存系统是其性能优化的核心,支持多层存储自动分层:

存储层次结构

S2D 按照性能自动识别存储介质类型:

  1. NVMe - 最高性能层
  2. SSD - 中等性能层
  3. HDD - 容量层

缓存配置表

存储介质组合 缓存层 容量层 缓存类型 说明
全 NVMe 部分 NVMe 剩余 NVMe 仅写缓存(可选) 全闪存,超高性能
全 SSD 部分 SSD 剩余 SSD 仅写缓存(可选) 全闪存,高性能
NVMe + SSD NVMe SSD 仅写缓存 混合闪存配置
NVMe + HDD NVMe HDD 读写缓存 高性能混合存储
SSD + HDD SSD HDD 读写缓存 标准混合存储
NVMe+SSD+HDD NVMe SSD+HDD 混合缓存 三层存储架构

缓存策略详解

仅写缓存 (Write-Only Cache)

适用场景:全闪存配置

写入流程:
应用写入 → NVMe缓存 → 后台刷新 → SSD容量层
读取流程:
应用读取 → 直接从SSD容量层

特点

  • 写入被缓存以提高 IOPS 和延迟
  • 读取直接访问容量层(因为都是闪存,性能足够)
  • 减少不必要的数据重复

读写缓存 (Read+Write Cache)

适用场景:混合存储配置

写入流程:
应用写入 → 快速介质缓存 → 后台刷新 → HDD容量层
读取流程:
应用读取 → 缓存命中? → 是:从缓存返回 / 否:从HDD读取并缓存

特点

  • 热数据保留在快速介质中
  • 为 HDD 提供去随机化 (De-randomization)
  • 显著提升混合存储的整体性能

缓存类比

将 S2D 缓存比作秘书的桌面缓冲系统:

  • NVMe 缓存 = 秘书的桌面:最常用文件立即可得
  • SSD 容量层 = 办公桌抽屉:需要时快速获取
  • HDD 容量层 = 文件柜:大容量存储,访问较慢
  • 缓存策略 = 秘书的工作习惯:决定什么文件放在哪里

7. 驱动器绑定机制 (Drive Binding)

S2D 采用动态的缓存驱动器绑定策略,而非固定的 1:1 绑定。

绑定规则

最小要求:每个服务器至少 2 个缓存驱动器 绑定比例:1:1 到 1:12+ (动态调整)

示例配置:
节点 A: 2x NVMe (缓存) + 8x HDD (容量)
绑定方案: NVMe#1 ←→ HDD#1,#2,#3,#4
         NVMe#2 ←→ HDD#5,#6,#7,#8

动态负载均衡

S2D 会根据以下因素动态调整绑定:

  • 各缓存驱动器的负载情况
  • 容量驱动器的访问模式
  • 缓存命中率和性能指标
  • 驱动器健康状态

绑定优势

相比固定绑定,动态绑定提供:

  • 更好的负载分布
  • 故障时更快的恢复
  • 更高的整体性能
  • 简化的管理复杂度

8. 缓存驱动器故障处理 (Cache Drive Failure Handling)

当缓存驱动器发生故障时,S2D 有完整的容错和恢复机制。

故障检测与响应

  1. 立即检测:驱动器故障被立即检测到
  2. 写入保护:停止向故障驱动器写入新数据
  3. 数据安全:已写入的数据在其他节点存在副本
  4. 自动重构:开始数据重构过程

未刷新写入的处理

问题:缓存中可能有未刷新到容量层的写入数据

解决方案

  • S2D 的分布式特性确保写入数据在多个节点存在副本
  • 即使单个缓存驱动器故障,数据不会丢失
  • 自动从其他副本恢复数据

重新绑定过程

故障检测 → 隔离故障驱动器 → 重新计算绑定 → 数据重构 → 恢复正常
    ↓            ↓              ↓           ↓         ↓
  瞬间         瞬间           几秒钟      几分钟    完全恢复

临时状态:在重新绑定期间,存储池会处于”Unhealthy”状态,但仍然可用。

最佳实践

  • 监控告警:配置缓存驱动器健康监控
  • 快速替换:尽快替换故障的缓存驱动器
  • 预防性维护:定期检查驱动器健康状态
  • 充足配置:确保有足够的缓存驱动器容量

9. 与其他缓存的关系 (Relationship with Other Caches)

S2D 环境中存在多层缓存,需要正确配置以避免冲突:

Storage Spaces 写回缓存

建议:在 S2D 环境中不要修改此缓存设置

  • 由 S2D 自动管理
  • 手动调整可能导致性能下降
  • 默认设置已针对 S2D 优化

CSV 内存读缓存

位置:运行在每个集群节点的内存中 功能:缓存经常访问的文件系统元数据和数据

配置建议

# 查看当前配置
Get-ClusterSharedVolumeState

# 调整缓存大小 (谨慎操作)
# (Get-ClusterSharedVolume "Volume1").SharedVolumeInfo.BlockCacheSize

缓存层次结构

应用程序
    ↓
文件系统缓存 (页面缓存)
    ↓
CSV 内存读缓存
    ↓
S2D 缓存层 (NVMe/SSD)
    ↓
物理存储介质 (HDD/SSD)

每层缓存都有其特定用途,相互协作提供最佳性能。

10. 要求与最佳实践 (Requirements & Best Practices)

最小系统要求

节点数量:最少 2 个节点,推荐 3-16 个节点 网络:10GbE 以上,强烈推荐 25GbE 或更高 RDMA:推荐使用 iWARP、RoCE 或 InfiniBand 内存:每节点至少 32GB,推荐 64GB+

网络配置最佳实践

# 启用 SMB Direct
Set-SmbServerConfiguration -EnableSMBDirectOverEthernet $true

# 优化 RDMA 设置
Set-NetAdapterAdvancedProperty -Name "Ethernet1" -RegistryKeyword "NetworkDirect" -RegistryValue 1

# 验证 RDMA 功能
Get-SmbServerNetworkInterface | Where-Object RdmaCapable -eq $true

存储配置建议

缓存层建议

  • 缓存容量至少为工作集大小的 10%
  • NVMe 优于 SSD,SSD 优于 HDD
  • 每个节点至少 2 个缓存驱动器以确保冗余

容量层建议

  • 使用企业级驱动器
  • 相同型号和容量的驱动器以获得最佳性能
  • 考虑故障域分布

监控与维护

关键监控指标

  • 存储池健康状态
  • 缓存命中率
  • 网络吞吐量和延迟
  • 驱动器健康状态

常用监控命令

# 检查存储池状态
Get-StoragePool
Get-VirtualDisk

# 监控 S2D 性能
Get-StorageQoSPolicy
Get-Counter "\Cluster Storage Hybrid Disks(*)\*"

# 验证网络性能
Test-SRTopology

故障排查工具

主要工具

  • Get-StorageHealthAction - 存储健康建议
  • Get-StorageJob - 当前存储操作状态
  • PrivateCloud.DiagnosticInfo - 综合诊断信息收集
  • Test-Cluster - 集群验证测试

Deep Dive: Storage Spaces Direct (S2D) Internals

Topic: Storage Spaces Direct (S2D) Architecture and Internal Implementation
Category: Storage
Level: Expert / Advanced
Last Updated: 2026-03-17


1. Overview

Storage Spaces Direct (S2D), introduced in Windows Server 2016, represents a fundamental shift in software-defined storage (SDS) architecture. Unlike traditional Windows failover clusters that rely on shared SAS storage or SAN devices, S2D uses only local disks on each node and constructs a distributed storage pool through high-speed networking.

Core Philosophy: Transform the traditional “shared storage” model into a “distributed local storage” model. If traditional SAN is like “multiple families sharing one big garage,” then S2D is like “each family having their own garage, connected through pneumatic tubes (RDMA) for resource sharing.” This architecture eliminates dependency on expensive shared storage hardware while providing better scalability and performance.

The core value of S2D lies in simplifying storage architecture, reducing costs, and achieving enterprise-class storage high availability, fault tolerance, and performance optimization through software-defined approaches. It eliminates the need for traditional SCSI-3 Persistent Reservations (PR) because all coordination is handled through Cluster Service and the Storage Bus Layer.

2. Shared SAS vs S2D Architecture Comparison

Traditional Shared SAS Architecture

In Windows Server 2012 R2 and earlier versions, failover clusters used shared storage architecture:

[Node1] ──┬── SAS/FC ──┬── [Shared Enclosure]
[Node2] ──┘            ├── RAID Controller
[Node3]                ├── Disk Arrays
[NodeN]                └── SCSI-3 PR Locking

Characteristics:

  • Each node connects to common storage devices via SAS/FC
  • Requires dedicated storage network (SAN)
  • Depends on SCSI-3 Persistent Reservation to prevent data corruption
  • Storage scaling limited by storage controllers and rack space
  • Single point of failure risk (storage controller)

S2D Distributed Architecture

S2D fundamentally changes this model:

[Node1-Local Storage] ──┐
[Node2-Local Storage] ──┼── 10GbE+ RDMA Network ──┐
[Node3-Local Storage] ──┤                         ├── Software-Defined Pool
[NodeN-Local Storage] ──┘                         └── Distributed Mirror/Parity

Characteristics:

  • Each node uses only locally connected disks
  • Connected via 10GbE+ Ethernet (RDMA recommended)
  • No shared storage hardware required
  • Linear scaling: adding nodes increases both compute and storage
  • No single point of failure (distributed architecture)
  • No SCSI-3 PR required

3. Storage Bus Layer (SBL): The Heart of S2D

The Storage Bus Layer (SBL) is the core component of S2D, creating a “virtual shared SAS bus” that makes distributed local storage appear like traditional shared storage.

SBL Component Architecture

Local Node                Remote Node
┌─────────────┐           ┌─────────────┐
│ Spaceport   │           │ Spaceport   │
├─────────────┤           ├─────────────┤
│ ClusPort.sys│ ←──RDMA──→ │ClusBflt.sys │
│ (Initiator) │           │ (Target)    │
├─────────────┤           ├─────────────┤
│ Local Disks │           │ Local Disks │
└─────────────┘           └─────────────┘

ClusPort.sys (SBL Initiator)

Function: Responsible for initiating storage requests to other cluster nodes

  • Operates as a SCSI Miniport driver
  • Forwards local storage I/O requests to remote nodes
  • Transports block device I/O via SMB Direct (RDMA)
  • Maintains connection pools to other nodes

ClusBflt.sys (SBL Target)

Function: Responds to storage requests from other nodes

  • Operates as a Storage Bus Filter driver
  • Receives and processes remote node I/O requests
  • Implements SBL Cache (read/write caching mechanism)
  • Manages local disk access control

Hidden SMB Shares

SBL creates hidden SMB shares in the background:

\\Server\BlockStorage$\Device\1
\\Server\BlockStorage$\Device\2

Characteristics:

  • Specifically for block device access, not file sharing
  • Transports SCSI commands via SMB3 protocol
  • Supports SMB Direct (RDMA) optimization
  • Completely transparent to users and applications

Block-over-SMB3 Protocol

SBL implements an innovative “Block over SMB3” protocol:

  • SCSI commands encapsulated in SMB3 messages
  • Data transfer optimized through SMB Direct (RDMA)
  • Maintains SCSI semantics for upper layer driver compatibility
  • Analogy: Transmitting SCSI bus signals through Ethernet pipes

4. Complete S2D Storage Stack

The S2D storage stack is more complex than traditional storage, involving two passes through partition manager and disk drivers:

Physical Disk Path

Application
    ↓
ReFS/NTFS
    ↓
CSV (Cluster Shared Volume)
    ↓
Volume Manager
    ↓
Partmgr.sys (Second Pass)
    ↓
Disk.sys (Second Pass)
    ↓
Spaceport.sys ← Virtual Disk Creation
    ↓
ClusBflt.sys (SBL Target)
    ↓
ClusPort.sys (SBL Initiator)
    ↓
Partmgr.sys (First Pass)
    ↓
Disk.sys (First Pass)
    ↓
Storport.sys
    ↓
Miniport Driver (e.g., stornvme.sys)
    ↓
Physical Disk Hardware

Reason for Double Pass

First Pass: Handle physical disks

  • Physical disks processed by Storport → Miniport → Disk.sys → Partmgr
  • SBL layer gains raw disk access rights
  • Creates distributed storage pool

Second Pass: Handle virtual disks

  • Virtual disks created by Spaceport re-enter the storage stack
  • Pass through Disk.sys → Partmgr → Volume Manager again
  • Finally used by CSV and file systems

This design ensures complete compatibility between S2D virtual disks and traditional disks at upper layers.

5. Spaceport.sys: Storage Spaces Manager

Spaceport.sys is the core driver for Storage Spaces, serving critical functions in S2D:

Primary Responsibilities

  1. Disk Claiming
    • Identifies disks available for storage pools
    • Excludes system disks, small capacity disks, etc.
    • Manages disk lifecycle
  2. Storage Pool Management
    • Creates and maintains distributed storage pools
    • Cross-node disk aggregation
    • Pool health monitoring and maintenance
  3. Virtual Disk Creation
    • Creates virtual disks according to fault tolerance policies
    • Supports different resiliency types like mirroring and parity
    • Dynamic allocation and thin provisioning
  4. Fault Tolerance Handling
    • Detects disk and node failures
    • Automatic data reconstruction and repair
    • Maintains multiple data copies

SpaceDump.sys

This is Spaceport’s debugging companion driver:

  • Collects storage state information during system crashes (BSOD)
  • Generates detailed storage pool diagnostic data
  • Helps Microsoft support teams analyze storage-related issues

6. S2D Caching Architecture

S2D’s caching system is the core of its performance optimization, supporting multi-tier storage auto-tiering:

Storage Tier Hierarchy

S2D automatically identifies storage media types by performance:

  1. NVMe - Highest performance tier
  2. SSD - Medium performance tier
  3. HDD - Capacity tier

Cache Configuration Matrix

Storage Media Mix Cache Tier Capacity Tier Cache Type Description
All NVMe Part NVMe Remaining NVMe Write-only (optional) All-flash, ultra-high performance
All SSD Part SSD Remaining SSD Write-only (optional) All-flash, high performance
NVMe + SSD NVMe SSD Write-only Hybrid flash configuration
NVMe + HDD NVMe HDD Read+Write High-performance hybrid storage
SSD + HDD SSD HDD Read+Write Standard hybrid storage
NVMe+SSD+HDD NVMe SSD+HDD Mixed cache Three-tier storage architecture

Cache Strategy Details

Write-Only Cache

Use Case: All-flash configurations

Write Flow:
Application Write → NVMe Cache → Background Flush → SSD Capacity Tier
Read Flow:
Application Read → Direct from SSD Capacity Tier

Characteristics:

  • Writes are cached to improve IOPS and latency
  • Reads access capacity tier directly (because all flash provides sufficient performance)
  • Reduces unnecessary data duplication

Read+Write Cache

Use Case: Hybrid storage configurations

Write Flow:
Application Write → Fast Media Cache → Background Flush → HDD Capacity Tier
Read Flow:
Application Read → Cache Hit? → Yes: Return from Cache / No: Read from HDD and Cache

Characteristics:

  • Hot data retained in fast media
  • Provides de-randomization for HDDs
  • Significantly improves overall hybrid storage performance

Cache Analogy

Think of S2D caching as a secretary’s desktop buffer system:

  • NVMe Cache = Secretary’s desktop: Most frequently used files immediately available
  • SSD Capacity Tier = Desk drawers: Quick access when needed
  • HDD Capacity Tier = File cabinets: Large capacity storage, slower access
  • Cache Policy = Secretary’s work habits: Decides which files go where

7. Drive Binding Mechanism

S2D adopts dynamic cache drive binding policies rather than fixed 1:1 binding.

Binding Rules

Minimum Requirement: At least 2 cache drives per server Binding Ratio: 1:1 to 1:12+ (dynamically adjusted)

Example Configuration:
Node A: 2x NVMe (cache) + 8x HDD (capacity)
Binding Scheme: NVMe#1 ←→ HDD#1,#2,#3,#4
               NVMe#2 ←→ HDD#5,#6,#7,#8

Dynamic Load Balancing

S2D dynamically adjusts binding based on:

  • Load conditions of each cache drive
  • Access patterns of capacity drives
  • Cache hit rates and performance metrics
  • Drive health status

Binding Advantages

Compared to fixed binding, dynamic binding provides:

  • Better load distribution
  • Faster recovery during failures
  • Higher overall performance
  • Simplified management complexity

8. Cache Drive Failure Handling

When cache drives fail, S2D has complete fault tolerance and recovery mechanisms.

Failure Detection and Response

  1. Immediate Detection: Drive failures are detected immediately
  2. Write Protection: Stop writing new data to failed drives
  3. Data Safety: Written data exists as copies on other nodes
  4. Automatic Reconstruction: Begin data reconstruction process

Handling Unflushed Writes

Problem: Cache may contain write data not yet flushed to capacity tier

Solution:

  • S2D’s distributed nature ensures write data exists as copies on multiple nodes
  • Even if a single cache drive fails, data is not lost
  • Automatically recovers data from other replicas

Rebinding Process

Failure Detection → Isolate Failed Drive → Recalculate Binding → Data Reconstruction → Return to Normal
       ↓                   ↓                     ↓                    ↓                    ↓
   Instant              Instant              Few seconds          Few minutes         Full recovery

Temporary State: During rebinding, the storage pool will be in “Unhealthy” state but remains available.

Best Practices

  • Monitoring Alerts: Configure cache drive health monitoring
  • Quick Replacement: Replace failed cache drives as soon as possible
  • Preventive Maintenance: Regularly check drive health status
  • Adequate Configuration: Ensure sufficient cache drive capacity

9. Relationship with Other Caches

Multiple cache layers exist in S2D environments and need proper configuration to avoid conflicts:

Storage Spaces Write-Back Cache

Recommendation: Do not modify this cache setting in S2D environments

  • Automatically managed by S2D
  • Manual adjustments may cause performance degradation
  • Default settings are optimized for S2D

CSV In-Memory Read Cache

Location: Runs in memory on each cluster node Function: Caches frequently accessed file system metadata and data

Configuration Recommendations:

# View current configuration
Get-ClusterSharedVolumeState

# Adjust cache size (use caution)
# (Get-ClusterSharedVolume "Volume1").SharedVolumeInfo.BlockCacheSize

Cache Hierarchy

Applications
    ↓
File System Cache (Page Cache)
    ↓
CSV In-Memory Read Cache
    ↓
S2D Cache Layer (NVMe/SSD)
    ↓
Physical Storage Media (HDD/SSD)

Each cache layer serves specific purposes and works together to provide optimal performance.

10. Requirements & Best Practices

Minimum System Requirements

Node Count: Minimum 2 nodes, recommended 3-16 nodes Networking: 10GbE or higher, strongly recommended 25GbE or higher RDMA: Recommended iWARP, RoCE, or InfiniBand Memory: At least 32GB per node, recommended 64GB+

Network Configuration Best Practices

# Enable SMB Direct
Set-SmbServerConfiguration -EnableSMBDirectOverEthernet $true

# Optimize RDMA settings
Set-NetAdapterAdvancedProperty -Name "Ethernet1" -RegistryKeyword "NetworkDirect" -RegistryValue 1

# Verify RDMA capability
Get-SmbServerNetworkInterface | Where-Object RdmaCapable -eq $true

Storage Configuration Recommendations

Cache Tier Recommendations:

  • Cache capacity should be at least 10% of working set size
  • NVMe preferred over SSD, SSD preferred over HDD
  • At least 2 cache drives per node for redundancy

Capacity Tier Recommendations:

  • Use enterprise-grade drives
  • Same model and capacity drives for optimal performance
  • Consider fault domain distribution

Monitoring and Maintenance

Key Monitoring Metrics:

  • Storage pool health status
  • Cache hit rates
  • Network throughput and latency
  • Drive health status

Common Monitoring Commands:

# Check storage pool status
Get-StoragePool
Get-VirtualDisk

# Monitor S2D performance
Get-StorageQoSPolicy
Get-Counter "\Cluster Storage Hybrid Disks(*)\*"

# Verify network performance
Test-SRTopology

Troubleshooting Tools

Primary Tools:

  • Get-StorageHealthAction - Storage health recommendations
  • Get-StorageJob - Current storage operation status
  • PrivateCloud.DiagnosticInfo - Comprehensive diagnostic information collection
  • Test-Cluster - Cluster validation tests

References

由于这是基于微软官方文档和技术规范的综合性技术文章,主要参考了微软的 Windows Server 技术文档、Storage Spaces Direct 官方指南等资料。本文内容基于对 S2D 技术的深度理解和实际部署经验整理而成。

References are based on Microsoft official Windows Server technical documentation, Storage Spaces Direct official guides, and practical deployment experience with S2D technology.