Uncategorized
VMware vSphere Host Path Limit
The VMware vSphere Host storage path limit as of vSphere 5.5 is 1024 and the maximum LUNs per host is 256. Maximum LUNS PER ESXi HOST = 256 Maximum FC […]
The VMware vSphere Host storage path limit as of vSphere 5.5 is 1024 and the maximum LUNs per host is 256.
Maximum LUNS PER ESXi HOST = 256
Maximum FC LUN ID =255
FC Number of paths to a LUN = 32
Number of total paths on a server = 1024
For more Max & Mins see @Virten‘s post: ‘VMware vSphere ESX and vCenter Configuration Maximums’
So what exactly does this mean; for example if a host consists of two FC HBA’s with HBA-0 connecting to Fabric-A and HBA-1 connecting to FabricB, as per the following diagram:
As can be seen this example results in 4 paths (2 per fabric) from the ESXi host to the target storage, which is a common configuration and satisfies most workload requirements. Taking into consideration the storage maximums outlined above this configuration allows you to connect exactly 256 storage devices (not taking into consideration local storage) from the attached SAN (Xtremio in this case :-)) to your ESXi host. VMware supports up to 256 FC LUNs and 1024 storage paths per host – That assumes a max of 4 paths per LUN:
1024 / 4 = 256 Possible LUNs
VMware supports a maximum of 32 paths to an FC LUN. If you did 32 paths per LUN that would result in 1024/32 = 32 possible LUNs presented before you start having issues. Some issues that may result if exceeding this limit are: VMware rescans – Rescans taking a very long time and/or causing hosts to crash and possible LUN disconnects.
Please revert to your respective storage vendors documentation on best practices for the number of paths to use. While using more than the required number of paths restricts the number of LUNs you can present to a host, it may also affect host resources such as CPU cycles and also every path from a host requires an initiator record and there may be a limit to the number of initiators per storage system.
Large numbers of paths i.e in excess of 4 per host may be required in environments where bursts of I/O from other hosts sharing the storage system ports are unpredictable and severe, for example boot storm scenarios where 8 paths per host might be plausible.
Viewing the Number of Active Paths
From the VI Client navigate to >Configuration >Storage Adapters and select the HBA:
Each HBA in this example has 512 paths – The maximum number of supported paths of 1024 has been reached.
Using a PowerCLI script to view the number of paths per ESXi Host (Thanks to @LucD22 ):
Connect-VIServer -Server 10.10.10.1 -User administrator@vsphere.local -password Password
$esx = Get-VMHost "HostName"
foreach($hba in (Get-VMHostHba -VMHost $esx -Type "FibreChannel"))
{
$target = ($hba.VMhost.ExtensionData.Config.StorageDevice.ScsiTopology.Adapter | where {$_.Adapter -eq $hba.Key}).Target
$luns = Get-ScsiLun -Hba $hba -LunType "disk"
$nrPaths = ($target | %{$_.Lun.Count} | Measure-Object -Sum).Sum
Write-Host $hba.Device "Targets:" $target.Count "Devices:" $luns.Count "Paths:" $nrPaths
}
If the maximum number of paths is exceeded you will see the following entry in the ESXi host logs:
Or examine the logs using PowerCLI for the entry: “The maximum number of supported paths of 1024 has been reached.”
Get-VIEvent $esx | Where {$_.FullFormattedMessage -match "The maximum number of supported paths of 1024 has been reached."} | Select CreatedTime,FullFormattedMessage
Note: neither the UI or CLI will return a value that exceeds the maximum supported configuration. For example the HBA path count will not increment above 512 for the example above if we add an additional device. The logs are the best way to check for the exceeded limitation – afaik.
VMWare KB1020654- Maximum number of supported paths reached
Impact
Storage paths that are discovered after the first 1024 are not added to the system. You will not have access to some of the storage paths, and possibly some storage devices, that are configured to be accessible by the system.
DavidRing.ie 