In this section:
The following sections contain VM performance tuning recommendations to improve system performance. These performance recommendations are general guidelines and are not exhaustive.
Ribbon recommends the following BIOS settings for optimum performance. The following table is intended as a reference. Exact values may vary based on vendor and HW.
Recommended BIOS Settings for Optimum Performance
Parameter | Recommended Setting |
---|---|
CPU power management/ Power Regulator | Maximum performance, or Static High Performance |
Intel Hyper-Threading | Enabled |
Intel Turbo Boost | Enabled |
Intel VT-x (Virtualization Technology) | Enabled |
Thermal Configuration | Optimal Cooling, or Maximum Cooling |
Minimum Processor Idle Power Core C-state | No C-states |
Minimum Processor Idle Power Package C-state | No Package state |
Energy Performance BIAS | Max Performance |
Sub-NUMA Clustering | Disabled |
HW Prefetcher | Disabled |
SRIOV | Enabled |
Intel® VT-d | Enabled |
BIOS Setting Recommendations for HP DL380p Gen8 Server
Parameter | Ribbon Recommended Settings | Default Settings |
---|---|---|
HP Power Profile | Maximum Performance | Balanced Power and Performance |
Thermal Configuration | Maximum Cooling | Optimal Cooling |
HW Prefetchers | Disabled | Enabled |
Adjacent Sector Prefetcher | Disabled | Enabled |
Processor Power and Utilization Monitoring | Disabled | Enabled |
Memory Pre-Failure Notification | Disabled | Enabled |
Memory Refresh Rate | 1x Refresh | 2x Refresh |
Data Direct I/O | Enabled | Disabled |
SR-IOV | Enabled | Disabled |
Intel® VT-d | Enabled | Disabled |
Log in to the ESX host
Check the NIC in use by using the esxcli network nic list
command.
Find out the NUMA affinity of the NIC using the command vsish -e get /net/pNics/<vmnicx>/properties | grep "NUMA".
Use the VMware vSphere client to configure the following ESXi host configuration parameters on the Advanced Settings page (see figure below) before installing the SBC SWe.
Path: Host > Manage > Advanced Settings
ESXi Advanced Settings
ESXi Parameter | ESXi 5.1 Settings | ESXi 5.5 Settings | ESXi 6.0 Settings | ESXi 6.5 Settings | ESXi 6.7 Settings | |||||
Recommended | Default | Recommended | Default | Recommended | Default | Recommended | Default | Recommended | Default | |
Cpu.CoschedCrossCall | 0 | 1 | 0 | 1 | 0 | 1 | 0 | 1 | 0 | 1 |
Cpu.CreditAgePeriod | 500 | 3000 | 1000 | 1000 | 1000 | 1000 | 1000 | 3000 | 1000 | 3000 |
DataMover.HardwareAcceleratedInit | 0 | 1 | 0 | 1 | 0 | 1 | 0 | 1 | 0 | 1 |
DataMover.HardwareAcceleratedMove | 0 | 1 | 0 | 1 | 0 | 1 | 0 | 1 | 0 | 1 |
Disk.SchedNumReqOutstanding | 256 | 32 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a |
Irq.BestVcpuRouting | 1 | 0 | 1 | 0 | 1 | 0 | 1 | 0 | 1 | 0 |
Mem.BalancePeriod | 0 | 15 | 0 | 15 | n/a | n/a | n/a | n/a | n/a | n/a |
Mem.SamplePeriod | 0 | 60 | 0 | 60 | n/a | n/a | n/a | n/a | n/a | n/a |
Mem.ShareScanGHz | 0 | 4 | 0 | 4 | 0 | 4 | 0 | 4 | 0 | 4 |
Mem.VMOverheadGrowthLimit | 0 | 4294967295 | 0 | 4294967295 | 0 | 4294967295 | 0 | 4294967295 | 0 | 4294967295 |
Misc.TimerMaxHardPeriod | 2000 | 100000 | 2000 | 100000 | 2000 | 100000 | 2000 | 500000 | 2000 | 500000 |
Misc.TimerMinHardPeriod | 100 | 100 | n/a | n/a | n/a | n/a | n/a | n/a | n/a | n/a |
Net.AllowPT | 1 | 0 | 1 | 0 | 1 | 0 | 1 | 1 | 1 | 1 |
Net.MaxNetifRxQueueLen | 500 | 100 | 500 | 100 | 500 | 100 | n/a | n/a | n/a | n/a |
Net.MaxNetifTxQueueLen | 1000 | 500 | 1000 | 500 | 1000 | 500 | 1000 | 2000 | 2000 | 2000 |
Net.NetTxCompletionWorldlet | 0 | 1 | 0 | 1 | n/a | n/a | n/a | n/a | n/a | n/a |
Net.NetVMTxType | n/a | n/a | n/a | n/a | n/a | n/a | 1 | 2 | 1 | 2 |
Net.NetTxWordlet | 0 | 2 | 1 | 2 | 1 | 2 | n/a | n/a | n/a | n/a |
Numa.LTermFairnessInterval | 0 | 5 | 0 | 5 | 0 | 5 | 0 | 5 | 0 | 5 |
Numa.MonMigEnable | 0 | 1 | 0 | 1 | 0 | 1 | 0 | 1 | 0 | 1 |
Numa.PageMigEnable | 0 | 1 | 0 | 1 | 0 | 1 | 0 | 1 | 0 | 1 |
Numa.PreferHT | 1 | 0 | 1 | 0 | 1 | 0 | 1 | 0 | 1 | 0 |
Numa.RebalancePeriod | 60000 | 2000 | 60000 | 2000 | 60000 | 2000 | 60000 | 2000 | 60000 | 2000 |
Numa.SwapInterval | 1 | 3 | 1 | 3 | 1 | 3 | 1 | 3 | 1 | 3 |
Numa.SwapLoadEnable | 0 | 1 | 0 | 1 | 0 | 1 | 0 | 1 | 0 | 1 |
To edit CPU, go to Edit instance settings > Virtual Hardware > CPU.
Edit CPU Settings Screen
Recommended CPU Settings
Parameter | Recommended Settings |
---|---|
Cores Per Socket | 1 |
Reservation | Value = (No. of vCPUs * CPU frequency)/2 or example, No. of vCPUs associated with the SBC = 32 CPU Model : Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz Hence Value = 36800 MHz ( i.e., 32 x 2300 / 2) |
Limit | Unlimited |
Shares | Normal |
To edit Memory, go to Edit instance settings > Virtual Hardware > Memory.
Edit Memory Settings
Recommended Memory Settings
Parameter | Recommended Settings |
---|---|
RAM | As per requirement. |
Reservation | Same as RAM. Check "Reserve all guest memory (All locked)" |
Limit | Unlimited |
Shares | Normal |
To edit Latency sensitivity, go to Edit instance settings > VM Options > Advanced > Latency sensitivity.
Latency sensitivity settings
Configure the VM Latency Sensitivity to High, if the ESX allows it.
To edit NUMA settings, go to Edit instance settings > VM Options > Advanced > Configuration Parameters > Edit Configuration.
Configure numa.nodeAffinity based on the NUMA node to which Pkt NICs are attached (as mentioned in General Recommendations). Ensure the VM memory fits in a single NUMA node, so that remote memory access does not happen.
NUMA Settings numa.nodeAffinity
NUMA Parameter Settings numa.vcpu.preferHT
Configure numa.autosize.once = FALSE.
NUMA.autosize