Clustersize, Blocksize, and Allocation Unit Size

Depending on where you see the terms used in Windows, you may see different names for the same thing. The clustersize of a volume, the blocksize of a volume, and the allocation unit size of the volume are all referring to the same value. Some tools report the blocksize as the physical blocksize, instead of the volume blocksize, which can lead to confusion between those terms. In this article, we will use the term clustersize.

Regardless of whether your disk volumes are for holding Exchange databases and logfiles, SQL databases and logfiles, Hyper-V VMs, or other large files; there is something you should do in order to properly optimize your access to those files. Adjust your clustersize. While we focus on Exchange here, most of the same ideas and concepts apply to many other applications.

The clustersize is used for several different things. What we care about:

  • The Master File Table (MFT) allocates storage on a disk volume based on the clustersize (that is, when a program requests to extend a file, the file is extended in multiples of the clustersize).
  • Exchange reads and writes to a disk volume based on multiples of the clustersize. Exchange also attempts to keep clusters contiguous.
  • Exchange allocates storage for logfiles and databases based on multiples of the clustersize. For example, a logfile is 1 MB in size (1 048 576 bytes). Exchange will allocate the 1 MB for a logfile all at once. If your clustersize is 4 096 (4 KB, the default) then the file consumes 256 clusters. If your clustersize is 65 536 (64 KB), then the file consumes 16 clusters. It is far more likely that 16 clusters can be allocated contiguously than 256 clusters.
  • Disk fragmentation is based on the contiguous or discontiguous placement of clusters.

The Exchange Server preferred architecture (PA) has long stated that logfiles and databases should reside on disk volumes where the clustersize is 64 KB. This is also true for the SQL Server PA. In earlier releases of Windows Server, it was also necessary to ensure that the initial location on a physical disk occur at a multiple of 64 KB. Beginning with Windows Server 2008 (??, I think), the OS automatically begins the initial allocation at the 1 MB boundary (which is an even multiple of 64 KB, as shown above).

In order to create a 64 KB volume, you can use whatever tool you prefer: format.com, diskpart.exe, WMI/CIM, or PowerShell’s Format-Volume. Or, if you don’t need to script this, you can use the GUI.

However, determining the clustersize isn’t the easiest thing in the world. There is no obvious place to find this value in the GUI. There is no obvious place to find this in PowerShell Get-* cmdlets (Get-Volume and Get-Partition do not return this information). If you want to go old-school, you use fsutil.exe in this manner:

PS C:\> fsutil.exe fsinfo ntfsinfo D:
NTFS Volume Serial Number :       0x2edec3c5dec38395
NTFS Version   :                  3.1
LFS Version    :                  2.0
Number Sectors :                  0x0000000c473befff
Total Clusters :                  0x00000000188e77df
Free Clusters  :                  0x00000000188e69ce
Total Reserved :                  0x0000000000000000
Bytes Per Sector  :               512
Bytes Per Physical Sector :       512
Bytes Per Cluster :               65536
Bytes Per FileRecord Segment    : 1024
Clusters Per FileRecord Segment : 0
Mft Valid Data Length :           0x0000000000010000
Mft Start Lcn  :                  0x000000000000c000
Mft2 Start Lcn :                  0x0000000000000001
Mft Zone Start :                  0x000000000000c000
Mft Zone End   :                  0x000000000000cca0
Resource Manager Identifier :     5012D937-F3EB-11E4-80BF-549F35094798
PS C:\>

And we see our answer in “Bytes Per Cluster”. If this is NOT the value we see, then we need to re-format the drive.

However, there is certainly a way to obtain this using PowerShell (or VBScript, for that matter). We switch to Windows Management Instrumentation (WMI). You perform the query this way:

PS C:\> Get-WmiObject -Class Win32_Volume  | ft -auto Label, Blocksize, Name

Label           Blocksize Name
-----           --------- ----
System Reserved      4096 \\?\Volume{998bc8e0-edd0-11e4-80b5-806e6f6e6963}
Data                65536 D:
                     4096 C:
PS C:\>

Again, if our data volume does not have a value of 64 KB, then we will need to reformat it.

Why the big deal about a larger clustersize? Well, it’s about efficiency (reducing fragmentation and optimizing I/O) and – perhaps surprisingly – that with low values of clustersize, it is possible to have a disk with lots of empty space – but files can no longer grow on the disk volume. This is because of an “artifact” of the MFT called the FAL (the File Attribute List). The FAL has a fixed maximum size of 256 KB. Among other things, the FAL keeps track of how many clusters are assigned to a given file. With the limit of 256 KB, the FAL can “only” store about one-and-a-half million fragmented clusters. On a volume with 4 KB clusters, the FAL will run out of space around at well less than 1 TB (depending on the overall fragmentation level of the volume). I have seen this occur with a customer on a database volume with a database size of less than 400 GB.

There used to be a KB article discussing this issue, but I can no longer find it online. Perhaps your favorite search engine can help you locate it. 🙂 KB 967351.

Exchange 2016 supports ReFS, which has a fixed clustersize of 64 KB (nothing smaller, nothing larger). According to recent postings about Exchange 2016, the Exchange 2016 Preferred Architecture will use ReFS instead of NTFS.

Please follow me on Twitter: @essentialexch

Postscript:
You’ll notice in the PowerShell example above that C:\ does not have a Label associated with it. In Windows Server 2012 R2, this is easily fixed in PowerShell:

PS C:\> Set-Volume -DriveLetter C -NewFileSystemLabel System
PS C:\> Get-WmiObject -Class Win32_Volume  | ft -auto Label, Blocksize, Name

Label           Blocksize Name
-----           --------- ----
System Reserved      4096 \\?\Volume{998bc8e0-edd0-11e4-80b5-806e6f6e6963}
Data                65536 D:
System               4096 C:
PS C:\> 

 

DPM 2010 Backing Up SQL 2012

I can't speak as to whether this is supported or not, but if you want DPM 2010 to be able to back up a SQL 2012 database instance, then you need to make a simple security change. Otherwise, DPM 2010 reports that "The replica is not consistent."

In words, NT AUTHORITY\System must be granted Sysadmin permissions on the database instance. NT AUTHORITY\System, by default, has public permissions on the instance in SQL 2012. However, in earlier versions of SQL, NT AUTHORITY\System has Sysadmin permissions at installation.

Below, the first figure identifies the specific login in the SQL Server Management Studio that has to be modified. The second figure identifies the box that needs to be checked. After checking the box, click OK to commit the change.

After this, you can open the DPM Console and for all of those SQL 2012 databases that are not consistent, go to the Protection workspace, right-click on the database(s), and then click on "Perform Consistency Check".

Figure 1 – The NT AUTHORITY\System login

Figure 2 – Setting the Sysadmin Server Instance Role