This section provides the server, operating system and host configuration required to run the DSC virtual machines.
Performance cannot be guaranteed for customer provided servers that have a different hardware configuration than the Ribbon Dell R740 provided server.
If physical separation of flows is not a requirement then these can be added to the same physical interface and have logical separation using VLAN. This is only supported on interfaces operating at 10G or higher speeds.
Although not all of the 1.2TB is used it is recommend to engineer these sizes to allow for growth in future releases.
These can be SATA or SAS drives.
Balanced memory configurations enable optimal interleaving which maximizes memory bandwidth. Ensure the memory provides balanced configuration by properly configuring the memory channels across the memory controller.
When the memory subsystem is incorrectly configured, the memory bandwidth available to the server can become limited and overall server performance can be reduced.
Dell Specific
Dell Recommended
These NIC models are known to cause outages when SR-IOV is configured with RHEL 8, and are therefore unsupported.
The following actions are needed to prepare the server for the application VM install. This is not a comprehensive list but is meant to detail the key steps. Additional information is located in the Example Configuration.
Setup OAM Interface and IP address on server using the bond, vlan, bridge structure specified in this document for the office type; geo or co-located. For an example see the example in Example Configuration.
Bridge name: Geo office = br-oamA (siteA), or br-oamB (siteB), Co-located office = br-dataoam
The OS is based on RHEL-8.2-x86_64 or later with the Virtualization Host packages selected during the RHEL install in the Software section of the installation menu, with the additional Virtualization Platform and Container Management Packages.
The following additional packages are required:
If you are using a host linux distribution that does not have ifconfig or brctl support, the following additional packages are required when using the default networking templates provided in the DSC SWe package installer.
brige-utils
sshpass is available via the RHEL "extras" repository. It is also available directly from the fedoraproject web site using these commands:
yum localinstall sshpass-1.06-9.el8.x86_64.rpm
The currently available version may be different than the one shown in this example.
The following packages are recommended:
In a customer supplied server solution the customer is responsible for the host OS installation, RHEL license, security updates,, integration in the customer back office for fault and alarms and for any host or operating system issues work directly with RedHat and/or the server vendor for support.
It is also critical that the customer implements a backup strategy in order to be able to recover the DSC in the event of a disk or server failure. The backup images must be stored off board.
Host Engineering Rules
The following BIOS configuration must be enabled on the host
How to Set on Dell R740 Servers
BIOS settings
Some BIOS may have different System Profile options and will require the profile to be configured in the OS. Refer to the "Example configuration" section below for more details.
NIC Settings
Recommendation is to create a large LVM. The VM images are stored by default in /var/lib/libvirt/images.
Although not all of the 1.2TB is used it is recommend to engineer this server with 1.2TB to allow for growth in future releases.
Recommended Disk Configuration
Mount Point | Volume Name | Size | Usage |
---|---|---|---|
/home | home | 50G | Used for following directories:
|
/opt | opt | 10G | Used for following directories
|
/ | root | 10G | |
/boot | boot | 1G | |
/boot/efi | 512M | ||
/tmp | tmp | 10G | |
/var | var | 10G | |
/var/log | var_log | 10G | For logs |
/var/log/audit | var_log_audit | 4G | Audit Logs |
/var/tmp | var_tmp | 10G | |
/backups | backups | 50G | Used for ReaR Backup. Following directories are created:
|
/upgrades | upgrades | 100G | Reserved for future use. |
/var/lib/libvirt/images | var_lib_libvirt_images | All Remaining Space | Used for VM disk storage (Example: qcow2 disks for vSP2K or DSC SWe) |
For proper reliability each server should be connected across two Layer 3 switches for co-located deployments and across two different interface modules in geo deployments. These ports will form an active/standby bond on the host.
VLAN tagging is required and multiple bridges are needed for the DSC solution. It is highly recommended for all DSC servers to have the same networking configuration.
For ansible to work the Host IP address must be configured on a bridge. Ansible does not support the host IP directly on an Ethernet interface.
The following network ports must be accessible on the host. Additional ports may be required based on the configuration.
Application/Service | Protocol | Port | Access Requirement |
---|---|---|---|
REAR - rpc | TCP | 111 | Other DSC server |
REAR - nfs | TCP | 2049 | Other DSC server |
REAR - mountd | TCP | 20048 | Other DSC server |
DNS | UDP/TCP | 53 | |
WebSM | TCP | 9090 | Allow cockpit |
ICMP | ICMP | Accept ICMP input packets | |
SSH/SFTP | TCP | 22 | |
Chronyd | UDP | 323 |
In a customer provided server the VM placement is engineered based on the number of vCPU, memory and disk available per server. These rules apply:
The Ribbon configurator provides for default recommended placement.
The following are general recommendations for a DSC SWe installation:
Allocate each VM with the required virtual hardware for robust operation. Provisioning a VM with more resources than it requires can, in some cases, reduce the performance of the VM and other VMs sharing the same KVM Host.
The following tables show the recommended resources allocated to each VM installed on a KVM Host.
Recommended KVM Host Resources Allocated to each DSC SWe VM for Diameter
Resources | To support Diameter only |
---|---|
vCPU | 4 cores; 2 GHz minimum |
vRAM | 5 GB |
vHDD | 65 GB |
vNIC | 4 virtual NICS (1 MGMT, 1 HA, and 2 packet ports) |
Recommended KVM Host Resources Allocated to each DSC SWe VM for SS7
Resources | To support SS7 Only |
---|---|
vCPU | 4 cores; 2 GHz minimum |
vRAM | 5 GB |
vHDD | 65 GB |
vNIC | 4 virtual NICS (1 MGMT, 1 HA, and 2 packet ports) |
Recommended KVM Host Resources Allocated for DSC (Diameter and SS7) VMs
Resources | 2 VMs supporting Diameter and SS7 |
---|---|
vCPU | 8 cores; 2 GHz minimum |
vRAM | 5 GB |
vHDD | 65 GB |
vNIC | 4 virtual NICS (1 MGMT, 1 HA, and 2 packet ports) |
Ribbon provides a utility called C20hostchecker which is available for download from GSC. This will validate that the server has the proper hardware and software installed to proceed with the virtual C20 application installation.
The c20hostchecker utility can be downloaded via Ribbon’s Global Software Center (GSC) website.
The software bundle downloaded from Ribbon’s GSC site contains the following items:
Refer to the user guide for details on how to modify the yaml configuration file and run the c20hostchecker utility. The results of the c20hostchecker must be shared with the Ribbon Network Engineered assigned to this project.
The information below is provided as job aids. Refer to redhat official documentation pages for extra details on host installation and network configuration located at https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8
Create bond, bridge, vlan and setup IP address on the bridge. This example creates bond "bond0" from interfaces eno1 and eno2, adds a bridge "br-oamA", and attaches the bridge to the bond using VLAN "bond0.172" with VLAN ID = 172.
# add the bonds nmcli c a type bond con-name bond0 ifname bond0 bond.options "mode=active-backup, miimon=100, fail_over_mac=follow" ipv4.method disable ipv6.method ignore # add the bond-slaves nmcli c a type bond-slave con-name eno1 ifname eno1 master bond0 nmcli c a type bond-slave con-name eno2 ifname eno2 master bond0 # add the bridges nmcli c a type bridge con-name br-oamA ifname br-host ipv4.method disable ipv6.method ignore # add the vlans (references both bond and bridge to tie them together) nmcli c a type vlan con-name bond0.172 ifname bond0.172 dev bond0 id 172 master br-oamA slave-type bridge # add the host IP nmcli c mod br-oamA ipv4.method manual ipv4.addresses 172.27.218.70/25 ipv4.gateway 172.27.218.1 ipv4.dns "172.27.218.216,172.27.218.116" ipv4.dns-search "example.com" # bounce the OAM bridge to cause the IP to be instantiated. nmcli c down br-oamA nmcli c up br-oamA
# Edit /etc/default/grub and add intel_iommu=on to GRUB_CMDLINE_LINUX. Example seen below GRUB_CMDLINE_LINUX="crashkernel=auto resume=/dev/mapper/rhel-swap rd.lvm.lv=rhel/root rd.lvm.lv=rhel/swap rhgb quiet intel_iommu=on" # Refresh the grub config change grub2-mkconfig -o /boot/grub2/grub.cfg # Reboot server reboot
# tuned-adm profile latency-performance
Ribbon recommends 50GB of disk space to be allocated for ReaR backup creation and storage. The disk space is recommended to be used for the following:
The ReaR ISO image should include both the bare-metal image and the backup data. Including the backup data with the bare-metal ISO image ensures the host image and data align. This format also eliminates the need for network connectivity when restoring a server.
The directories listed in this section must be included in the backup, they contain tools, scripts, and configuration files necessary for recovering the DSC Virtual Machines.
Directory | Contents |
---|---|
/opt/rbbn | Ribbon scripts, tools, and configuration files. This is the Ribbon default location, refer to the site Network Specification Document (Specbook) for the Ribbon tools location. |
/var/lib/libvirt/swtpm | DSC VM configuration files, including the XML files. |
/var/lib/libvirt/network | |
/var/lib/libvirt/filesystems | |
/var/lib/libvirt/boot | |
/var/lib/libvirt/dnsmasq | |
/var/lib/libvirt/qemu | |
/etc/libvirt/qemu |
Directories listed in this section must be excluded from the backup to prevent the backup from failing.
Directory | Contents |
---|---|
/var/lib/libvirt/images | DSC VM disk images. The images are large and constantly changing. |
Rear backup directory | The host backup should not include the mate, or local, backup images. |
/var/log | The system logs are not needed for restore. |
This section identifies the file types that should be excluded from the backup to prevent the backup from getting to large. Including these file types may cause the backup to fail. A large backup ISO image may also slow recovery.
Directory | Comment |
---|---|
Installation ESD, ISO and QCOW2 images | The files are large and may cause the backup to fail. It is recommended to download installation images from GSC when required. |
Patches | It is recommended to re-download patches after a system recovery. |
This section provides a recommendation for the ReaR configuration file. It is recommended that the /etc/rear/local.conf file be used for the default ReaR server configuration. These variables override the default ReaR variable definitions in the /usr/share/rear/conf/default.conf file.
Variable Definition Recommendations for the local.conf file.
Variable | Description |
---|---|
OUTPUT=ISO | Identifies the output file type. ISO is the only supported output type. |
BACKUP=NETFS | The internal backup method. |
OUTPUT_URL=nfs:<nfs mount> | Identifies the locate of the backup. It is recommended to use an NFS mount to the mate server to allow for multiple backup copies to be maintained. |
BACKUP_URL=iso:///backup/ | The BACKUP_URL specifies where the backup is located during the restore. This definition directs ReaR to place the backup data in the ISO backup image. |
export TMPDIR="<dir>" | This defines the ReaR working directory. In some systems the /tmp directory does not allow executable files to be run, in which case an alternate directory is needed. |
KEEP_BUILD_DIR="" | This variable instructs ReaR to cleanup its build directory after every backup. This is needed to prevent ReaR from exceeding its disk limit. |
KEEP_OLD_OUTPUT_COPY=y | This variable specifies that two backups should be maintained, current and "old". At most, ReaR maintains two copies. This variable only applies when the backups are maintained on an NFS mount. |
BACKUP_PROG_EXCLUDE=( "${BACKUP_PROG_EXCLUDE[@]}" '/var/lib/libvirt/images/*' '...') | This variable specifies the directories that ReaR should excluded from the backup. |
The following table outlines the responsibility for Ribbon provided hardware or customer sourced hardware.
Component | Ribbon Provided Hardware | Customer Provided Hardware | ||||||||
Install | Support | Upgrade &Security Patches | Logs, Alarms North-bound | System Recovery | Install | Support | Upgrade, &Security Patches | Log and Alarms North-bound | System Recovery | |
DSC Applications | R | R | R1 | R | R | R | R6 | R1 | R | R/C5 |
VM Instantiation | R2 | R | R1 | R | R | R2 | R | R1 | R | R |
Operating System and Hypervisor | R | R | R1 | R | R | C3 | C3 | C3 | C3 | C3 |
Hardware | R | R | R1 | R | R | C4 | C4 | C4 | C4 | C4 |
Network (Router, CS LAN) | C | C | C | C | C | C | C | C | C | C |
R = Ribbon, C = Customer
Notes:
Customers under Ribbon Care may choose to self apply or have Ribbon apply corrective updates. All upgrades are available via Ribbon support portal and Ribbon Pro Services can be contracted on a per occurrence basis.
VM Created as part of Ansible playbook at commission
HOST OS must be provided by the customer and updates obtained directly from Red Hat (ie. HOST OS not under a Ribbon license). All required packages must be applied, supported NICs installed and disks portioned. Customer responsible for restoration in a failure.
Customer must maintain proper backups
Ribbon does not certify performance on customer provided hardware, including where hardware complies to Ribbon recommended specifications