Versions Compared

Old Version 1

changes.mady.by.user Ribbon Communications

Saved on

compared with

New Version 2

changes.mady.by.user Ribbon Communications

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Add_workflow_for_techpubs
AUTH1
JIRAIDAUTHSBX-104043
REV5
REV6
REV3
REV4
REV1
REV2



Panel

In this section:

Table of Contents
maxLevel4



Info
iconfalse

Related articles:

Children Display


The following sections contain VM performance tuning recommendations to improve system performance. These performance recommendations are general guidelines and are not exhaustive.

BIOS Setting Recommendations

Ribbon recommends the following BIOS settings for optimum performance. The following table is intended as a reference. Exact values may vary based on vendor and HW.

Caption
0Table
1Recommended BIOS Settings for Optimum Performance
3Recommended BIOS Settings for Optimum Performance


ParameterRecommended Setting

CPU power management/ Power Regulator

Maximum performance, or Static High Performance

Intel Hyper-ThreadingEnabled
Intel Turbo BoostEnabled
Intel VT-x (Virtualization Technology)Enabled
Thermal Configuration

Optimal Cooling, or Maximum Cooling

Minimum Processor Idle Power Core C-stateNo C-states
Minimum Processor Idle Power Package C-stateNo Package state
Energy Performance BIASMax Performance

Sub-NUMA Clustering

 Disabled
HW PrefetcherDisabled
SRIOVEnabled
Intel® VT-d

Enabled


The BIOS settings shown in the example below are recommended for HP DL380p Gen8 servers. For BIOS settings of other servers, refer to the respective vendor's website.

Caption
0Table
1BIOS Setting Recommendations for HP DL380p Gen8 Server
3BIOS Setting Recommendations for HP DL380p Gen8 Server


ParameterRibbon Recommended SettingsDefault Settings
HP Power Profile Maximum PerformanceBalanced Power and Performance
Thermal ConfigurationMaximum CoolingOptimal Cooling
HW PrefetchersDisabledEnabled
Adjacent Sector PrefetcherDisabledEnabled
Processor Power and Utilization MonitoringDisabledEnabled
Memory Pre-Failure NotificationDisabledEnabled
Memory Refresh Rate1x Refresh2x Refresh
Data Direct I/OEnabledDisabled
SR-IOVEnabledDisabled
Intel® VT-dEnabledDisabled


Anchor
General Recommendations
General Recommendations
General Recommendations

  • Ensure the number of vCPUs in an instance is always an even number (4, 6, 8, and so on), as hyper threaded vCPUs are used.
  • For best performance, make sure a single instance is confined to a single NUMA. Performance degradation occurs if an instance spans across multiple NUMAs.
  • Ensure the physical NICs associated with an instance are connected to the same NUMA/socket where the instance is hosted. Doing so reduces the remote node memory access, which in turn helps improve the performance.

    • Log in to the ESX host

    • Check the NIC in use by using the esxcli network nic list command.

    • Find out the NUMA affinity of the NIC using the command vsish -e get /net/pNics/<vmnicx>/properties | grep "NUMA".

ESXi Host Configuration Parameters

Use the VMware vSphere client to configure the following ESXi host configuration parameters on the Advanced Settings page (see figure below) before installing the SBC SWe.

Path: Host > Manage > Advanced Settings


Div
classpdf6pttext


Caption
0Table
1ESXi Advanced Settings


ESXi Parameter       ESXi 5.1 Settings      ESXi 5.5 Settings       ESXi 6.0 Settings      ESXi 6.5 Settings     ESXi 6.7 Settings

RecommendedDefaultRecommendedDefaultRecommendedDefaultRecommendedDefaultRecommendedDefault
Cpu.CoschedCrossCall0101010101
Cpu.CreditAgePeriod500300010001000100010001000300010003000
DataMover.HardwareAcceleratedInit0101010101
DataMover.HardwareAcceleratedMove0101010101
Disk.SchedNumReqOutstanding25632n/an/an/an/an/an/an/an/a
Irq.BestVcpuRouting1010101010
Mem.BalancePeriod015015n/an/an/an/an/an/a
Mem.SamplePeriod060060n/an/an/an/an/an/a
Mem.ShareScanGHz0404040404
Mem.VMOverheadGrowthLimit0429496729504294967295042949672950429496729504294967295
Misc.TimerMaxHardPeriod20001000002000100000200010000020005000002000500000
Misc.TimerMinHardPeriod100100n/an/an/an/an/an/an/an/a
Net.AllowPT1010101111
Net.MaxNetifRxQueueLen500100500100500100500n/an/an/an/a
Net.MaxNetifTxQueueLen1000500100050010005001000200020002000
Net.NetTxCompletionWorldlet0101n/an/an/an/an/an/a
Net.NetVMTxTypen/an/an/an/an/an/a1212
Net.NetTxWordlet021212n/an/an/an/a
Numa.LTermFairnessInterval0505050505
Numa.MonMigEnable0101010101
Numa.PageMigEnable0101010101
Numa.PreferHT1010101010
Numa.RebalancePeriod600002000600002000600002000600002000600002000
Numa.SwapInterval1313131313
Numa.SwapLoadEnable0101010101



VM Settings

CPU Settings

To edit CPU, go to Edit instance settings > Virtual Hardware > CPU.

Caption
0Figure
1Edit CPU Settings Screen
3Edit CPU Settings Screen

Recommended CPU Settings

Caption
0Table
1Recommended CPU Settings
3Recommended CPU Settings


ParameterRecommended Settings
Cores Per Socket 1 
Reservation 

Value = (No. of vCPUs * CPU frequency)/2

or example,

No. of vCPUs associated with the SBC = 32 

CPU Model : Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz

Hence Value = 36800 MHz  ( i.e., 32 x 2300 / 2)

LimitUnlimited
SharesNormal


Memory Settings

To edit Memory, go to Edit instance settings > Virtual Hardware > Memory.


Caption
0Figure
1Edit Memory Settings
3Edit Memory Settings

Recommended Memory Settings

Caption
0Table
1Recommended Memory Settings
3Recommended Memory Settings


ParameterRecommended Settings
RAMAs per requirement.
Reservation 

Same as RAM. 

Check "Reserve all guest memory (All locked)"

LimitUnlimited
SharesNormal


Latency Sensitivity Settings

To edit Latency sensitivity, go to Edit instance settings > VM Options > Advanced > Latency sensitivity.

Caption
0Figure
1Latency sensitivity settings
3Latency sensitivity settings


Configure the VM Latency Sensitivity to High, if the ESX allows it.

Info
titleNote
  • ESX 6.5 allows configuring latency sensitivity to High even with hyper-threaded CPU reservation.
  • ESX 6.7 and above does not allow configuring latency sensitivity to High without full CPU core reservation.

NUMA Settings

To edit NUMA settings, go to Edit instance settings > VM Options > Advanced > Configuration Parameters > Edit Configuration.

Configure numa.nodeAffinity based on the NUMA node to which Pkt NICs are attached (as mentioned in General Recommendations). Ensure the VM memory fits in a single NUMA node, so that remote memory access does not happen.

Caption
0Figure
1NUMA Settings numa.nodeAffinity
3NUMA Settings


Configure numa.vcpu.preferHT=TRUE. This is required for better cache optimizations. Refer to http://www.staroceans.org/ESXi_VMkernel_NUMA_Constructs.htm for further details.

Caption
0Figure
1NUMA Parameter Settings numa.vcpu.preferHT
3NUMA Parameter Settings numa.vcpu.preferHT


Configure numa.autosize.once = FALSE.

Caption
0Figure
1NUMA.autosize
3NUMA.autosize