Supported Deployment Scenarios
GPU acceleration is supported on SBC SWe cloud-based T-SBC instances on OpenStack (Newton and above). T-SBC is a component in a Distributed SBC architecture that provides transcoding service.
GPU devices are attached to SBC cloud instances through PCIe pass-through – a single GPU device can be used by only one instance at a time. The process of enabling PCIe pass-through in OpenStack is detailed later in this document. For performance considerations, NUMA locality of devices should be ensured.
| Info |
|---|
NVIDIA GRID is not supported. |
Supported GPU Devices
NVIDIA Tesla V100(PCIe), 16 GB variant only
Supported Codecs
- AMR-NB
- G729
- G722
- AMRWB
AMRNB- AMR-WB
In addition, G.711 is supported for GPU instances, but only when G.711 is being transcoded to a non-G.711 codec. You cannot currently configure transcoding from G.711 to G.711 on GPU instances. The coding rates and packetization times for the supported codecs are shown in the tables on the Audio Codecs page.
| Include Page |
|---|
| G.722_Silence_Suppression_Note |
|---|
| G.722_Silence_Suppression_Note |
|---|
|
| Info |
|---|
|
The following procedures assume that the supported GPU devices have been properly installed on the server. |
Instantiating GPU T-SBC on OpenStack Cloud
The T-SBC is instantiated using the help of a specific heat template. The GPU T-SBC requires a special flavor that has appropriate directives to utilize GPU devices of the compute node available for PCIe pass-through.
T-SBC Heat Template
The T-SBC instance should be launched using the heatRgNoDhcp-TSBC-template.yaml template. This template shares all fields of an Man M-SBC template, and additionally has the following fields:
| Caption |
|---|
| 0 | Table |
|---|
| 1 | T-SBC Heat Template |
|---|
|
| | |
|---|
gpu | Indicate whether to use GPU or CPU for transcoding. Should be set as true for GPU TSBCs. Note: For GPU T-SBCs ,additional provisioning of codec percentages are required at the time of instantiation. Transcode resources for codecs are reserved and fixed for the lifetime of the instance. | True | AMR | Percentage of channels to be allocated for AMR codec (0-100) Applicable only when gpu field is True. | 100 | AMRWB
| Percentage of channels to be allocated for AMRWB codec (0-100) Applicable only when gpu field is True. | 0 | EVRC (not applicable in this release
| Percentage of channels to be allocated for EVRC codec (0-100) Applicable only when gpu field is True. | 0 | EVRCB (not applicable in this release) | Percentage of channels to be allocated for EVRCB codec (0-100) Applicable only when gpu field is True. | 0 | G729
| Percentage of channels to be allocated for G729 codec (0-100) Applicable only when gpu field is True. | 0 | G722
| Percentage of channels to be allocated for G722 codec (0-100) Applicable only when gpu field is True. | 0 |
|
Host Changes on OpenStack for Enabling GPU Devices
This section describes the changes needed on the Controller node and the Compute node hosting GPU cards in order to enable instances to use GPU devices. While this section focuses purely on the GPU aspect, Ribbon recommends that you refer to broader OpenStack performance tuning recommendations covered in the following links:
| Caption |
|---|
| 0 | Table |
|---|
| 1 | OS Configuration for Compute Node with GPU Device |
|---|
|
| | |
|---|
| 1 | Edit /etc/sysconfig/grub and ensure that the following parameters are populated: intel_iommu=on iommu=pt rdblacklist=nouveau | - Enables kernel support for PCIe passthrough.
- Blacklists opensource NVIDIA driver (nouveau) from loading on the host.
| | 2 | Update grub using the following command: grub2-mkconfig -o /etc/grub2.cfg | | 3 | Create /etc/modprobe.d/nouveau.conf file with the following contents: blacklist nouveau | Blacklists opensource NVIDIA driver (nouveau) from loading on the host. | | 4 | Reboot the compute node. | |
|
| Caption |
|---|
| 0 | Table |
|---|
| 1 | Openstack Configuration for Compute Node with GPU Device |
|---|
|
| | |
|---|
| 1 | Add a PCI alias for the GPU device in /etc/nova/nova.conf. For V100: pci_alias={"vendor_id":"10de", "product_id":"1db4","device_type":"type-PCI","name":"v100gpu"}
Note: PCI alias will referred to by flavors that make use of this PCI device. | | 2 | Add a GPU device to existing the PCIe whitelist entries in /etc/nova/nova.conf. For V100: pci_passthrough_alias={"vendor_id":"10de", "product_id":"1db4","device_type":"type-PCI","name":"v100gpu"}whitelist=[{"devname": "p5p1", "physical_network": "sriov_1"}, {"devname": "p5p2", "physical_network": "sriov_2"},{"devname": "p6p1", "physical_network": "sriov_3"},{"devname": "p6p2", "physical_network": "sriov_4"}, {"vendor_id":"10de","product_id":"1db4"}]
Note: Whitelist PCI device for use in OpenStack. | | 3 | Restart nova-compute service: systemctl restart OpenStack-nova-compute.service
|
|
| Caption |
|---|
| 0 | Table |
|---|
| 1 | Openstack Configuration for Controller Node |
|---|
|
| | |
|---|
| 1 | Ensure PciPassthroughFilter and NumaTopologyFilter are added to scheduler_default_filters list in /etc/nova/nova.conf file. Note: Enables nova to instantiate instances with CPU resources from the same NUMA as that of the PCI devices to be used. | | 2 | Add PCI alias for the GPU device in /etc/nova/nova.conf file For V100: pci_alias={"vendor_id":"10de", "product_id":"1db4","device_type":"type-PCI", "name":"v100gpu"} Note:PCI alias will be referred to by flavors that make use of this PCI device | | 3 | Restart nova-api service systemctl restart OpenStack-nova-api.service |
|
Guideline for Creating Flavors for GPU T-SBC Instances
Open the dashboard and create a flavor with the following properties. Check with your Ribbon account team to determine the appropriate instance size for your traffic needs.
| Info |
|---|
|
This is a sample benchmark. Ribbon does not mandate the use of the processors shown here. |
| Caption |
|---|
| 0 | Table |
|---|
| 1 | Flavor Creation Guideline for GPU T-SBC Instances |
|---|
|
| |
|---|
| VCPUs | 20 | | RAM | 25 GiB | | Root Disk (min) | 65 GiB |
|
After creating the flavor, update its metadata with the following key values.
| Caption |
|---|
| 0 | Table |
|---|
| 1 | Metadata Key Values |
|---|
|
| | |
|---|
| hw:cpu_policy | dedicated | Ensures guest vCPUs are pinned to host cpus for performance. | | hw:numa_nodes | 1 | Ensures host cpus of a single NUMA node are used in the instance for performance. | | hw:cpu_thread_policy | prefer | This setting allocates each vCPU on thread siblings of physical CPUs. | | hw:cpu_max_sockets | 1 | This setting defines how KVM exposes the sockets and cores to the guest. | | pci_passthrough:alias | v100gpu:1 | Ensures NVIDIA PCIe devices are attached to the instance. |
|
Configuring the SBC for Invoking T-SBC
Refer to the following pages for basic configuration steps for the S-SBC and M-SBC:
For enabling T-SBC, some additional configurations are required in S-SBC and M-SBC which are described in subsequent sections.
Configuring and Activating T-SBC Cluster
Steps for configuration of T-SBC is similar to M-SBC with the following exception: The IP interface group creation procedure should create private interface groups instead of public. There are no public interface groups for T-SBC.
Additional Configuration for S-SBC
A DSP cluster needs to be configured in the S-SBC configuration to refer to the T-SBC cluster that is intended to be used for transcoding. The following steps describe the procedure for creation of this cluster:
| Caption |
|---|
| 0 | Table |
|---|
| 1 | Creating a DSP Cluster |
|---|
|
| Step | ActionNotes |
|---|
| 1 | Log on to the EMA and then click the All tab. | | | 2 | In the navigation pane, click System > DSBC > Cluster > Type and add the T-SBC node entry by selecting DSP:
 Image Removed
| | 3 | Click in the FQDN field and then add the corresponding FQDN for the T-SBC created in the T-SBC configuration. | | 4 | Click Save. | | | 5 | Refer to System Provisioning - Packet Service Profile for configuration changes that must be made on the S-SBC to enable transcoding. Note: In GPU T-SBCs, the required codecs and their percentages also need to must be provisioned in the Heat template as described in the previous section. This provisioning is fixed for the lifetime of the application. All members of a single T-SBC cluster should follow the same codec provisioning values. |
|
| Include Page |
|---|
| GPU_T-SBC_Codec_Restriction |
|---|
| GPU_T-SBC_Codec_Restriction |
|---|
|
Configure the private IP interface group for relaying media packets to T-SBC using steps from the "Configure Private LIF Groups in M-SBC" section of Invoke MRF as a Transcoder for D-SBC.
Licensing
All pre-existing licensing related to transcoding apply to GPU codecs as well. There is no separate license for GPU functionality.pagebreak