Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Fixed typo in warning
Panel

In this section:

Table of Contents
maxLevel4

 

Warning
titleWarning

'VLANs are not supported on SR-IOV PKT interfaces.

Prerequisite

Info
titleNote

 Ensure that SR-IOV is enabled on the BIOS settings of the host by logging in through iLO console.


Info
titleNote

SR- IOV is a licensed feature on the VMware and the "VMware vSphere Enterprise Plus" license must be procured to enable SR-IOV Support on the ESXi.

  • Install SR-IOV supported two 10 Gigabit PCI cards.

  • Minimum 10 GB RAM.
  • VM must be created with four Interfaces:
    • One virtual interface for Management Port

    • One virtual interface for HA Port

    • Two virtual interfaces for SR-IOV Ports

Info
titleNote

It is recommended to configure all four ports with different IP addresses in four different networks.

For example:

  • MGMT - Network 1
  • HA - Network 2
  • SR-IOV - Network 3
  • SR-IOV - Network 4

Configuring Virtual Machine Instances

Configuring SR-IOV

Perform the following steps:

  1. Log on to VMware ESXi GUI as the root user.

    Caption
    0Figure
    1VMWare ESXi 6.5

  2. To check the status of the SR-IOV cards:

    1. Navigate to Host > Manage.

    2. Select the tab Hardware.
      Ensure both the SR-IOV cards are in disabled state or the Passthrough must be in disabled state.

      Caption
      0Figure
      1SR-IOV Cards

  3. If the SR-IOV cards are not disabled, they must be disabled by performing following:

    1. Enable SSH. To enable SSH,

      1. Navigate to Host and select the tab Actions.

      2. From the drop-down list, select the option Services. Click Enable Secure Shell (SSH) and Enable console shell.

        Caption
        0Figure
        1Enable SSH

    2. Check the name of the NIC cards. To check the name of the NIC cards,

      1. Log on to CLI as root.

      2. Execute the following command:

        Code Block
        titleInput
        lspci|grep X540
        

        The following is the sample display output:

        Code Block
        0000:0b:00.0 Network controller: Intel(R) Ethernet Controller X540-AT2 [vmnic8]
        0000:0b:00.1 Network controller: Intel(R) Ethernet Controller X540-AT2 [vmnic9]
    3. Open the esx.conf file and find the PCI ID associated with the NIC card.

      1. Change the owner of the PCI ID from passthrough to vmkernel.

      2. Execute the following command:

        Code Block
        vi /etc/vmware/esx.conf
        

        The following is the sample display output:

        Code Block
        /system/uuid = "5a33649d-c9db-e792-c676-5cb9018acc24"
        /system/uservars/psa/defaultLunMasksInstalled = "1"  
        /system/uservars/corestorage/Filter/DefaultVMWRulesLoaded = "1"
        /system/uservars/corestorage/VAAI/DefaultVMWRulesLoaded = "1"  
        /system/uservars/host-acceptance-level = "partner"           
        /resourceGroups/version = "6.5.0"                 
        /adv/Misc/HostIPAddr = "10.54.12.81"
        /adv/Misc/DiskDumpSlotSize = "2560" 
        /adv/Misc/HostName = "hpg9-9"      
        /adv/Net/ManagementIface = "vmk0"
        /adv/Net/ManagementAddr = "10.54.12.81"
        /adv/UserMem/UserMemASRandomSeed = "1418738923"
        /adv/UserVars/HostClientCEIPOptIn = "1"        
        /device/00000:005:00.0/vmkname = "vmhba1"
        /device/00000:002:00.0/vmkname = "vmnic0"
        /device/00000:002:00.2/vmkname = "vmnic2"
        /device/00000:003:00.0/vmkname = "vmhba0"
        /device/00000:002:00.1/vmkname = "vmnic1"
        /device/00000:011:00.1/owner = "vmkernel"
        /device/00000:011:00.1/device = "1528"   
        /device/00000:011:00.1/vendor = "8086"
        /device/00000:011:00.1/vmkname = "vmnic9"
        /device/00000:004:00.2/vmkname = "vmnic6"
        /device/00000:004:00.1/vmkname = "vmnic5"
        /device/00000:002:00.3/vmkname = "vmnic3"
        /device/00000:004:00.0/vmkname = "vmnic4"
        /device/00000:005:00.1/vmkname = "vmhba2"
        /device/00000:004:00.3/vmkname = "vmnic7"
        /device/00000:011:00.0/vmkname = "vmnic8"
        /device/00000:011:00.0/vendor = "8086"   
        /device/00000:011:00.0/device = "1528"
        /device/00000:011:00.0/owner = "vmkernel"
        - /etc/vmware/esx.conf 33/499 6%
    4. Save the file.

    5. Reboot the host.

  4. In the VMware ESXi GUI, navigate to Host > Manage.

  5. Select the Hardware tab.

  6. From the PCI Devices, select the SR-IOV card.

    Caption
    0Figure
    1Selecting the SR-IOV Card

  7. Click Configure SR-IOV.

    The window to configure the SR-IOV card is displayed.

  8. For the option Enabled, select Yes and set the number of virtual functions.

    Caption
    0Figure
    1Configuring the SR-IOV Card

  9. Click Save.

  10. Configure the other SR-IOV card. Repeat the steps from 4 to 6.

  11. Reboot the host by clicking Reboot host.

    Caption
    0Figure
    1Reboot Host


    The following warning message is displayed.
    Caption
    0Figure
    1Warning Message

  12. Click Reboot.

    The VMware ESXi login window is displayed with the message "The host is rebooting...".

    Caption
    0Figure
    1The VMWare ESXI Login Window Displaying the Message

  13. Once the virtual function is created, the SR-IOV cards and the Passthrough for the virtual functions display the status as "Active".

    Caption
    0Figure
    1SR-IOV Cards and Virtual Function Status

Creating a vSwitch

Creating a Management vSwitch

Perform the following steps:

  1. Navigate to Networking. From the Port group tab, click the Add port group.
    The Add port group window is displayed.

    Caption
    0Figure
    1Adding the MGMT Port Group


    The following table describes the Add port group fields.
    Caption
    0Table
    1Add Port Group Fields
    FieldExample or Recommendation
    NameVMNetwork
    VLAN ID0
    Virtual switchvSwitch0
    SecuritySelect Inherit from vSwitch

  2. Click Add.

Creating an HA vSwitch

Perform the following steps:

  1. Navigate to Networking. From the Port group tab, click the Add port group.
    The Add port group window is displayed.

    Caption
    0Figure
    1Adding the HA Port Group


    The following table describes the Add port group fields.
    Caption
    0Table
    1Add Port Group Fields
    FieldExample or Recommendation
    NameHANetwork
    VLAN ID0
    Virtual switchvSwitch1
    SecuritySelect Inherit from vSwitch

  2. Click Add.

Creating the Virtual Machine

Perform the following steps:

  1. Navigate to Virtual Machines. Click Create / Register VM to create or register a virtual machine. The Select creation type option is displayed.

    Caption
    0Figure
    1Create or Register a Vrtual Machine

  2. Select the option Create a new virtual machine.

  3. Click Next.

    The Select a name and guest OS option is displayed.

    Caption
    0Figure
    1Select a Name and Guest OS


    The following table describes the Select a name and guest OS fields.
    Caption
    0Table
    1Select a Name and Guest OS Fields
    FieldExample or Recommendation
    NameName of the virtual machine. For example, VM.
    CompatibilityESXi 6.5 virtual machine
    Guest OS familyLinux
    Guest OS versionDebian GNULinux 8 (64-bit)

  4. Click Next.

    The Select storage option is displayed.

  5. Select datastore1.

    Info
    titleNote

    Ensure that the datastore has minimum 500 GB or more space. This datastore is required to store all log-related data files.

    Caption
    0Figure
    1Select Storage

  6. Click Next.
    The Customize settings option is displayed.

  7. Configure virtual hardware from Customize settings:

    1. Setting CPU

      When configuring virtual CPUs within the vSphere Web Client, you can configure:

      • The total number of vCPUs for the virtual machine
      • The total number of cores per socket

      The following table provides the examples of socket determination based on the CPU and Cores per Socket within the vSphere Web Client:

      Caption
      0Table
      1Number of Sockets Determined by the vSphere Web Client
      Total Number of virtual CPUs (CPU)Cores per SocketNumber of Sockets Determined by the vSphere Web Client
      441
      422
      414
      881
      824
      842
      818

      Info
      titleNote

      A minimum of 4 vCPUs is required. Any number of vCPUs may be configured depending upon the call capacity requirements, but the number should be even (4, 6, 8, etc.) to avoid impacting performance.

       

      Caption
      0Figure
      1Setting CPU


      The following table describes the CPU fields.
      Caption
      0Table
      1Customize Settings-CPU
      FieldsExample or Recommendation
      CPU10
      Number of Virtual sockets1
      Number of cores per virtual socket10
      CPU Reservation25850 MHz
      Limit25850 MHz
      SharesNormal
      Hardware virtualizationNone
      Performance countersNone
      Scheduling AffinityNone

    2. Setting Memory

      Caption
      0Figure
      1Setting Memory


      The following table describes the Memory fields.
      Caption
      0Table
      1Customize Settings-Memory
      FieldsExamples or Recommendation
      Memory20480 MB.
      Note: It is recommended to use more than 20 GB memory.
      ReservationSelect the option Reserve all guest memory (All locked)
      LimitUnlimited
      SharesNormal
      Memory Hot PlugNone

    3. Setting Hard disk 1

      Caption
      0Figure
      1Setting Hard disk 1


      The following table describes the Hard disk 1 fields.
      Caption
      0Table
      1Customize Settings - Hard disk 1
      FieldsExamples or Recommendation
      Hard disk 1200 GB
      Maximum Size1.43 TB
      Locationdatastore1
      Disk ProvisioningSelect Thick provisioned, lazily zeroed
      SharesNormal
      Limit-IOPsUnlimited
      Virtual Device NodeSCSI controller 0 and SCSI (0:0)
      Disk modeDependent
      SharingNone

    4. Setting SCSI Controller

      Caption
      0Figure
      1Setting SCSI Controller


      The following table describes the SCSI Controller fields.
      Caption
      0Table
      1Customize Settings - SCSI Controller
      FieldsExamples or Recommendation
      SCSI ControllerSelect LSI Logic Parallel from the drop-down list
      SCSI Bus SharingNone
      SATA Controller 0N/A
      USB controller 1USB 2.0

    5. Setting Network Adapter 1
      The Network Adapter 1 is used for provisioning MGMT ports.

      Caption
      0Figure
      1Setting Network Adapter 1


      The following table describes the Network Adapter 1 fields.
      Caption
      0Table
      1Customize Settings - Network Adapter 1
      FieldsExamples or Recommendation
      Network Adapter 1Select the MGMT ports from the drop-down list. For example, VM Network.
      StatusSelect the option Connect at power on.
      Adapter TypeVMXNET3
      MAC AddressAutomatic

      Once the Network Adapter 1 is created for MGMT ports, create a new Network Adapter for HA ports.

    6. Select Add network adapter. The option to create New Network Adapter for HA port is displayed.

      Caption
      0Figure
      1Add Network Adapter


      The following table describes the New Network Adapter fields.
      Caption
      0Table
      1Customize Settings - New Network Adapter
      FieldsExamples or Recommendation
      New Network AdapterSelect the HA port from the drop-down list. For example, HA Network.
      StatusSelect the option Connect at power on.
      Adapter TypeVMXNET3
      MAC AddressAutomatic

    7.  Click Add other device to continue the configuration on PKT0 and PKT1.

      Caption
      0Figure
      1Add Other Device

    8. Select the option PCI device from the drop-down list.

      Caption
      0Figure
      1Select PCI Device


      The New PCI device option is created.

    9. Repeat steps g and h to create one more PCI Device.

      Caption
      0Figure
      1New PCI device

    10. Click Next. The Ready to complete option is displayed.

      Caption
      0Figure
      1Ready to Complete

    11. Once the review is performed, click Finish. The Virtual Machine is created.

RSS Configuration for VMs using SR-IOV Interfaces

Receive side scaling (RSS) is a mechanism that enables spreading incoming network traffic across multiple CPUs, thus eliminating a potential processing bottleneck. For SR-IOV configurations, the RSS configuration can be updated in the ESXi host as follows:

 

On the ESXi host, execute the following command: 

esxcfg-module -s 'max_vfs=2,2,0,0 RSS=1,1' ixgbe

where: max_vfs is an array of integers representing number of virtual functions to be enabled for required devices and RSS is an array of integers used to enable RSS for the required devices.

Reboot the host to have RSS take effect.

Installing SBC on VMware

Once the VMware instance is created, you must install the SBC application.

Perform the following steps:

  1. Select the Virtual Machine where you want to install the SBC application.
  2. Click Edit. The Edit Settings window is displayed.

  3. Select CD/DVD Drive 1.

    Caption
    0Figure
    1Edit Settings

  4. The Datastore browser window is displayed. Browse the ISO image file.

    Caption
    0Figure
    1Selecting the OS File

  5. Click Select. The following window is displayed.

    Caption
    0Figure
    1Saving the OS File

  6. Click Save.
  7. Click Power on to power on the VM.

    Caption
    0Figure
    1Powering On the VM

  8. The Sonus SBC Installer window is displayed. Press Enter to boot.

    Caption
    0Figure
    1Sonus SBC Installer Window

  9. Once the installation completes, you are prompted to enter the login credentials.

    Caption
    0Figure
    1Login Information

  10. Log on to CLI as linuxadmin. Provide the following IP addresses:

    1. Primary Management IPv4 Address

    2. Primary Management Network Mask

    3. Primary Management Gateway IP Address, you are prompted to use IPv6 address.

    4. Enter n if you do not want to set IPv6

    Caption
    0Figure
    1Ip Addresses