Page History

Add_workflow_for_techpubs

AUTH1	UserResourceIdentifier{userKey=

8a00a0c86820e56901685f374974002d

8a00a0c87e188912017e4c24a00e0016, userName='null'}
JIRAIDAUTH	SBX-

104043

00000
REV5	UserResourceIdentifier{userKey=

8a00a02355cd1c2f0155cd26cb8305e9

8a00a0c87e188912017e4c24a00e0016, userName='null'}
REV6	UserResourceIdentifier{userKey=

8a00a02355cd1c2f0155cd26cb8305e9

8a00a02355cd1c2f0155cd26cd5909df, userName='null'}
REV3	UserResourceIdentifier{userKey=

8a00a02355cd1c2f0155cd26c752002f

8a00a02355cd1c2f0155cd26ce350b51, userName='null'}
REV4	UserResourceIdentifier{userKey=

8a00a02355cd1c2f0155cd26cba10618

8a00a02355cd1c2f0155cd26cdcd0ab1, userName='null'}
REV1	UserResourceIdentifier{userKey=

8a00a02355cd1c2f0155cd26ca1903b3

8a00a02355cd1c2f0155cd26cbfd06fb, userName='null'}

REV2UserResourceIdentifier{userKey=8a00a02355cd1c2f0155cd26c9f0036e, userName='null'}

Internal_display_only

Panel

In this section:

Table of Contents

maxLevel	4

Info

icon	false

Prerequisites

Info

title	Note

Ensure that SR-IOV is enabled on in the host's BIOS settings of the host by logging in through the iLO console.

Info

title	Note

When using SR-IOV interfaces, do do not add more than 64 VLANs, as the driver does not support it.

Info

title	Note

SR-IOV is a licensed feature on VMware. Procure the "VMware vSphere Enterprise Plus" license to enable SR-IOV support on ESXi.

Install SR-IOV supported two 10 Gigabit PCI cards.
Minimum 10 GB RAM.
VM must be created with four Interfaces:
- One virtual interface for Management Port
- One virtual interface for HA Port
- Two virtual interfaces for SR-IOV Ports

Info

title	Note

It is recommended to configure Configuring all four ports with different IP addresses in four different networks is recommended.

For example:

MGMT - Network 1
HA - Network 2
SR-IOV - Network 3
SR-IOV - Network 4

Configuring Virtual Machine Instances

Configuring SR-IOV

Perform the following steps:

Log on to VMware ESXi GUI as the root user.
caption
Image Added
0 Figure
1 VMWare ESXi 6.5
Image Removed
Include Page
Include Page
_Screenshot_disclaimer
_Screenshot_disclaimer
To check the status of the SR-IOV cards:
1. Navigate to Host > Manage.
2. Select the tab Hardware.
  Ensure both the SR-IOV cards are in a disabled state , or the Passthrough is in disabled state.
  Caption0
  
  Figure1SR-IOV Cards
  
  Image Modified

If the SR-IOV cards are not disabled, disable them by performing the following:

Enable SSH. To enable SSH,
1. Navigate to Host and select the tab Actions.
2. From the drop-down list, select the option Services. Click Enable Secure Shell (SSH) and Enable console shell.
  Caption0Figure
  
  1Enable SSH
  Image Modified
Check the name of the NIC cards. To check the name of the NIC cards,
1. Log on to CLI as root.
2. Execute Enter the following command:
  Code Block
  title Input
  lspci|grep X540
  The following is the sample display output:
  Code Block
  0000:0b:00.0 Network controller: Intel(R) Ethernet Controller X540-AT2 [vmnic8] 0000:0b:00.1 Network controller: Intel(R) Ethernet Controller X540-AT2 [vmnic9]

Open the esx.conf file and find the PCI ID associated with the NIC card.

Change the owner of the PCI ID from passthrough to vmkernel.

Execute Enter the following command:

Code Block
vi /etc/vmware/esx.conf

The following is the sample display output:

Code Block

/system/uuid = "5a33649d-c9db-e792-c676-5cb9018acc24"
/system/uservars/psa/defaultLunMasksInstalled = "1"  
/system/uservars/corestorage/Filter/DefaultVMWRulesLoaded = "1"
/system/uservars/corestorage/VAAI/DefaultVMWRulesLoaded = "1"  
/system/uservars/host-acceptance-level = "partner"           
/resourceGroups/version = "6.5.0"                 
/adv/Misc/HostIPAddr = "10.54.12.81"
/adv/Misc/DiskDumpSlotSize = "2560" 
/adv/Misc/HostName = "hpg9-9"      
/adv/Net/ManagementIface = "vmk0"
/adv/Net/ManagementAddr = "10.54.12.81"
/adv/UserMem/UserMemASRandomSeed = "1418738923"
/adv/UserVars/HostClientCEIPOptIn = "1"        
/device/00000:005:00.0/vmkname = "vmhba1"
/device/00000:002:00.0/vmkname = "vmnic0"
/device/00000:002:00.2/vmkname = "vmnic2"
/device/00000:003:00.0/vmkname = "vmhba0"
/device/00000:002:00.1/vmkname = "vmnic1"
/device/00000:011:00.1/owner = "vmkernel"
/device/00000:011:00.1/device = "1528"   
/device/00000:011:00.1/vendor = "8086"
/device/00000:011:00.1/vmkname = "vmnic9"
/device/00000:004:00.2/vmkname = "vmnic6"
/device/00000:004:00.1/vmkname = "vmnic5"
/device/00000:002:00.3/vmkname = "vmnic3"
/device/00000:004:00.0/vmkname = "vmnic4"
/device/00000:005:00.1/vmkname = "vmhba2"
/device/00000:004:00.3/vmkname = "vmnic7"
/device/00000:011:00.0/vmkname = "vmnic8"
/device/00000:011:00.0/vendor = "8086"   
/device/00000:011:00.0/device = "1528"
/device/00000:011:00.0/owner = "vmkernel"
- /etc/vmware/esx.conf 33/499 6%

Save the file.
Reboot the host.

In the VMware ESXi GUI, navigate to Host > Manage.
Select the Hardware tab.
From the PCI Devices, select the SR-IOV card.
Image Added
Click Configure
Caption0Figure1Selecting the
SR-IOV
Card
.
Image Removed
Click Configure SR-IOV.
The window to configure the SR-IOV card is displayed.
1
For the option Enabled, select Yes and set the number of virtual functions.
Caption0Figure

Configuring the SR-IOV Card
Image Modified
Click Save.
Configure the other SR-IOV card. Repeat the steps from 4 to 6.
Reboot the host by clicking Reboot host.
caption

0 Figure
1 Reboot Host
Image Modified
The following warning message is displayed.
caption

0 Figure
1 Warning Message
Image Modified
Click Reboot.
The VMware ESXi login window is displayed with the message displays "The host is rebooting...".
Caption
0 Figure
1 The VMWare ESXI Login Window Displaying the Message
Image Removed
Image Added
Once the Once the virtual function is created, the SR-IOV cards and the Passthrough for the virtual functions display the status as "Active".
Caption0Figure

1SR-IOV Cards and Virtual Function Status
Image Modified

Creating a vSwitch

Creating a Management vSwitch

Perform the following steps:

Navigate to Networking. From the Port group tab, click the Add port group.
The Add port group window is displayed .
Caption
0 Figure
1 Adding the MGMT Port Group
Image Removed
Include Page_Screenshot_disclaimer
with the following fields:
_Screenshot_disclaimerThe following table describes the Add port group fields. Caption
0 Table
1 Add Port Group Fields
FieldExample or RecommendationNameVMNetworkVLAN ID0
Field Example or Recommendation
Name VMNetwork
VLAN ID 0
Virtual switch vSwitch0
Security Select Inherit from vSwitch
Click Add.

Creating an HA vSwitch

Perform the following steps:

Navigate to Networking. From the Port group tab, click the Add port group.
The Add port group window is displayed .
Caption
0 Figure
1 Adding the HA Port Group
Image Removed
Include Page_Screenshot_disclaimer
with the following fields:
_Screenshot_disclaimerThe following table describes the Add port group fields. Caption
0 Table
1 Add Port Group Fields
Field
Field
Example or Recommendation
Name HANetwork
VLAN ID 0
Virtual switch vSwitch1
Security Select Inherit from vSwitch
Click Add.

Creating the Virtual Machine

Perform the following steps:

Navigate to Virtual Machines. Click Create / Register VM to create or register a virtual machine.
The Select creation type option is displayed.
caption
Image Added
0 Figure
1 Create or Register a Vrtual Machine
Image Removed
Include Page
_
Include Page
_Screenshot_disclaimer
_Screenshot_disclaimer
Select the option Create a new virtual machine.
Click Next.
The Select a name and guest OS option is displayed .
Caption
0 Figure
1 Select a Name and Guest OS
Image Removed
The following table describes the Select a name and guest OS fields. Caption
0 Table
1 Select a Name and Guest OS Fields
Field Example or Recommendation
Name Name of the virtual machine. For example, VM.
Compatibility ESXi 6.5 virtual machine
Guest OS family Linux
Guest OS version Debian GNULinux 8 (64-bit)
Click Next.
The Select storage option is displayed.

Select datastore1.

Info

title	Note

Ensure that the datastore has minimum 500 GB or more space. This datastore is required to store all log-related data files.

Caption

0	Figure
1	Select Storage

Image Removed

with the following fields:
Field Example or Recommendation
Name Name of the virtual machine. For example, VM.
Compatibility ESXi 6.5 virtual machine
Guest OS family Linux
Guest OS version Debian GNULinux 8 (64-bit)
Click Next.
The Select storage option is displayed.
Select datastore1.
Info
title Note
Ensure that the datastore has minimum 500 GB or more space. This datastore is required to store all log-related data files.
Image Added
Click Next.
The Customize settings option is displayed.
Configure virtual hardware from Customize settings:
1. Set the CPU
  When configuring virtual CPUs within the vSphere Web Client, you can configure:
  - The total number of vCPUs for the virtual machine
  - The total number of cores per socket
  The following table provides examples of socket determination based on the CPU and Cores per Socket within the vSphere Web Client:
  Total Number of virtual CPUs (CPU) Cores per Socket Number of Sockets Determined by the vSphere Web Client
  4 4 1
  4 2 2
  4 1 4
  8 8 1
  8 2 4
  8 4 2
  8 1 8
  
  Info
  title
Click Next.
The Customize settings option is displayed.

Configure virtual hardware from Customize settings:

Setting CPU

When configuring virtual CPUs within the vSphere Web Client, you can configure:

The total number of vCPUs for the virtual machine
The total number of cores per socket

The following table provides the examples of socket determination based on the CPU and Cores per Socket within the vSphere Web Client:

Caption

0	Table
1	Number of Sockets Determined by the vSphere Web Client

Total Number of virtual CPUs (CPU)

Cores per Socket

Number of Sockets Determined by the vSphere Web Client

4

1

4

2

4

1

4

8

1

8

2

4

8

4

2

8

1

8

Info

title	Note

A minimum of 4 vCPUs is required. Any number of vCPUs may be configured depending upon on the call capacity requirements, but the number should be even (4, 6, 8, etc.) to avoid impacting performance.

Caption

0	Figure
1	Setting CPU

Image Removed

Info

title	Note

Set the CPU reservation so that it equals to equal the physical processor CPU speed multiplied by the number of vCPUs assigned to the VM, divided by 2.

CPU Reservation = (No. of vCPUs * CPU frequency)/2

For example, a configuration of 32 CPUs with a processor of 2.30 GHz CPU frequency reserves "(32 * 2300)/2 = 36800 MHz".

The following table describes the CPU fields.

caption

0	Table
1	Customize Settings-CPU

Fields	Example or Recommendation
CPU	`10`
Number of Virtual sockets	`1`
Number of cores per virtual socket	`10`
CPU Reservation	`12925 MHz`
Limit	`Unlimited`
Shares	`Normal`
Hardware virtualization	`None`
Performance counters	`None`
Scheduling Affinity	`None`

Adjust the Memory using the following fields:
Fields Examples or Recommendation
Memory 20480 MB.
Note:
Setting Memory
Caption
0 Figure
1 Setting Memory
Image Removed
The following table describes the Memory fields. Caption
0 Table
1 Customize Settings-Memory
FieldsExamples or RecommendationMemory20480 MB.
Note:
It is recommended to use more than 20 GB of memory.
Reservation Select the option Reserve all guest memory (All locked)
Limit Unlimited
Shares Normal
Memory Hot Plug None

Setting Set the Hard disk 1

Caption

0	Figure
1	Setting Hard disk 1

Image Removed

.

The following table describes the Hard disk 1 fields.

Caption

0	Table
1	Customize Settings - Hard disk 1

FieldsExamples

Fields	Examples or Recommendation
Hard disk 1	`200 GB`
Maximum Size	`1.43 TB`
Location	`datastore1`
Disk Provisioning	Select `Thick provisioned, lazily zeroed`
Shares	`Normal`
Limit-IOPs	`Unlimited`
Virtual Device Node	`SCSI controller 0 and SCSI (0:0)`
Disk mode	`Dependent`
Sharing	`None`

0
Setting Set the SCSI Controller
Caption

Figure1Setting SCSI Controller
Image Removed
The following table describes the SCSI Controller fields.
Caption0Table
1Customize Settings - SCSI Controller
Fields Examples or Recommendation
SCSI Controller Select LSI Logic Parallel from the drop-down list
SCSI Bus Sharing None
SATA Controller 0 N/A
USB controller 1 USB 2.0

Setting Set the Network Adapter 1
The Network Adapter 1 is used for provisioning to provision MGMT ports.

Caption

0	Figure
1	Setting Network Adapter 1

Image Removed

The following table The following table describes the Network Adapter 1 fields.

Caption0Table

1Customize Settings - Network Adapter 1

Fields	Examples or Recommendation
Network Adapter 1	Select the MGMT ports from the drop-down list. For example, `VM Network`.
Status	Select the option `Connect at power on`.
Adapter Type	`VMXNET3`
MAC Address	`Automatic`

Once the Network Adapter 1 is created for MGMT ports, create a new Network Adapter for HA ports.

0
Select Add network adapter. The option to create a New Network Adapter for the HA port is displayed.
Caption

Figure1Add Network Adapter
Image Removed
The following table describes the New Network Adapter fields.
caption
Fields
0 Table
1 Customize Settings - New Network Adapter
FieldsExamples or RecommendationNew Network AdapterSelect
Examples or Recommendation
New Network Adapter Select the HA port from the drop-down list. For example, HA Network.
Status Select the option Connect at power on.
Adapter Type VMXNET3
MAC Address Automatic
To attach packet ports on VMware ESXi 7.0 and above versions, skip the steps 7.g to 7.j, and follow the steps in the section Attach SR-IOV Interface in VMware ESXi 7.0 and Above.
Anchor7.g7.gClick Add other device to continue the configuration on PKT0 and PKT1.
Caption0

Figure1Add Other Device
Image Modified
Select the option PCI device from the drop-down list.
Caption
0 Figure
1 Select PCI Device
Image Modified
The New PCI device option is created.

Repeat steps g and h to create one more PCI Device.

Caption

0	Figure
1	New PCI device

Image Removed

more PCI Devices from PKT networks.

Info

title	Note

Follow the same procedure for both a non-port redundancy and port redundancy scenario.

2 PKT interfaces are supported for a non-port redundancy scenario
4 PKT interfaces are supported for a port redundancy scenario

Ensure packet ports are attached to the SBC VM in the correct sequential order. See Readjust SR-IOV Network Adapter Addition Sequence.

Image Added

Anchor7.j7.jClick Next. The Ready to complete option is displayed.
Caption0Figure

1Ready to Complete
Image Modified
PKT Port NUMA Affinity: Perform the following steps:
1. Find the PKT port NUMA affinity by executing the following command on the EXSi ESXi host:
  Code Block
  vsish -e get /net/pNics/<PKT port name - vmnicX>/properties | grep "NUMA"
2. Update VM NUMA affinity to be the same as NIC NUMA affinity:
  1. Edit Settings -> VM options -> Configuration Parameters -> Add Parameters.
    Caption
    0 Figure
    1 Edit Configuration Parameters
    Add Parameters.
    Image Modified
  2. Add the following parameters:
    Add Parameters
    Code Block
    numa.nodeAffinity' = 0 or 1 (based on PKT port NIC affinity) numa.autosize.once = FALSE
    Caption
    0 Figure
    1
    Image Modified
Once the review is performed, click Finish. The Virtual Machine is created.

RSS Configuration for VMs using SR-IOV Interfaces

Receive side scaling (RSS) is a mechanism that enables spreading incoming network traffic across multiple CPUs, thus eliminating a potential processing bottleneck. For SR-IOV configurations, update the RSS configuration in the ESXi host as shown in the procedure below:

Execute Enter the following command to unload the IXGBE driver:
Code Block
esxcfg-module -u ixgbe
Execute Enter the following command to verify the driver is unloaded:
Code Block
esxcfg-module -l | grep ixg
Execute Enter one of the following commands to reload the driver with the required virtual function (VF) and RSS configurations:
Code Block
vmkload_mod ixgbe max_vfs=2,2 RSS=4,4
~ or ~
Code Block
esxcfg-module -s 'max_vfs=2,2, RSS=4,4' ixgbe
Reboot the host for the RSS to take effect. This modified configuration is retained after the reboot.

Installing the SBC Application on VMware

Once the VMware instance is created, install the SBC application.

Perform the following steps:

Select the Virtual Machine where you want to install the SBC application.
Click Edit. The Edit Settings window is displayed.
Select CD/DVD Drive 1.
Caption
0 Figure
1 Edit Settings
Image Modified
Include Page
_Screenshot_disclaimer
_Screenshot_disclaimer
0
The Datastore browser window is displayed. Browse the ISO image file.
Caption

Figure1Selecting the OS File
Image Modified
Click Select. The following window is displayed.
Caption
0 Figure
1 Saving the OS File
following window is displayed.
Image Modified
Click Save.
1Powering On the VM
Click Power on to power on the VM.
Caption0Figure

Image Modified
The SBC Installer window is displayed. Press Enter to boot.
Caption
0 Figure
1 SBC Installer Window
Image Removed
Once the installation completes, you are prompted to enter the login credentials.
Caption
0 Figure
1 Login Information
Image Modified
Log on to CLI as linuxadmin. Provide the following IP addresses:
1. Primary Management IPv4 Address
2. Primary Management Network Mask
3. Primary Management Gateway IP Address, you are prompted to use IPv6 address.
4. Enter n if you do not want to set IPv6
Caption0Figure1Ip Addresses
1. Image Modified

Space shortcuts

Page tree

Versions Compared

Old Version 2

New Version 3

Key

Prerequisites

Configuring Virtual Machine Instances

Configuring SR-IOV

Creating a vSwitch

Creating a Management vSwitch

Creating an HA vSwitch

Creating the Virtual Machine

RSS Configuration for VMs using SR-IOV Interfaces

Installing the SBC Application on VMware

Virtual switch	`vSwitch0`
Security	Select `Inherit from vSwitch`

Example or Recommendation
Name	`HANetwork`
VLAN ID	`0`
Virtual switch	`vSwitch1`
Security	Select `Inherit from vSwitch`

Field	Example or Recommendation
Name	Name of the virtual machine. For example, `VM`.
Compatibility	`ESXi 6.5 virtual machine`
Guest OS family	`Linux`
Guest OS version	`Debian GNULinux 8 (64-bit)`

It is recommended to use more than 20 GB of memory.
Reservation	Select the option `Reserve all guest memory (All locked)`
Limit	`Unlimited`
Shares	`Normal`
Memory Hot Plug	`None`

Fields	Examples or Recommendation
SCSI Controller	Select `LSI Logic Parallel` from the drop-down list
SCSI Bus Sharing	`None`
SATA Controller 0	`N/A`
USB controller 1	`USB 2.0`

Examples or Recommendation
New Network Adapter	Select the HA port from the drop-down list. For example, `HA Network`.
Status	Select the option `Connect at power on`.
Adapter Type	`VMXNET3`
MAC Address	`Automatic`

Code Block
numa.nodeAffinity' = 0 or 1 (based on PKT port NIC affinity) numa.autosize.once = FALSE
Caption
0	Figure
1