In this section:
You can upgrade the VM nodes in an SBC SWe VNF to a new version of software using VNFM if the nodes were instantiated using VNFM. Within VNFM, you specify the version of software to which you want to upgrade, select an upgrade method, and then upgrade the VM nodes beginning with the node that you determine currently has the standby role. To avoid service disruption you should only upgrade VM nodes when they are in standby mode.
The following procedure supports upgrading SBC N:1 HA deployments, where a maximum 4:1 HA deployment includes 5 VM instances (four active, one standby). However the process is similar regardless of the number of nodes in the cluster.
Due to release 9.0 changes made in OAM function to reduce the number of interfaces required on an OAM VM,
A compute host hardware change is not supported during an upgrade. A new VNFC instantiation is needed to do so.
Prerequisites
Before initiating the upgrade ensure the following:
If your deployment includes a
Unable to show "metadata-from": No such page "_space_variables"system, upgrade it before upgrading OAM nodes (if present) and the SBC nodes.
Refer to VNFM page, Upgrading a VNF, and the Ribbon Application Management Platform Documentation for additional details.- Download the required files for the new version of SBC software. Refer to the release notes for the new SBC software release. The required files include:
- .qcow2 file that contains the SBC software
- Script to generate a customized Cloud Service Archive (CSAR) package file
- Virtual Network Function Descriptor (VNFD) template file to use with the script
- Run the script to generate a custom CSAR package file for your deployment. Refer to Creating a CSAR Package File.
- Create an OpenStack Glance image using the SBC application software .qcow2 file. Refer to Creating a Glance Image.
- Onboard the CSAR package file you created for your deployment. Refer to Onboarding the SBC CSAR package file.
- Check the VNF Lifecycle window. The VNF you want to upgrade must be in Ready status, meaning that the status of each VM within it is Available.
- Ensure the pre-upgrade cluster configuration is saved to Unable to show "metadata-from": No such page "_space_variables". Note that for N:1 HA M-SBC upgrades, the Load Balancing Service (LBS) must be configured on the nodes prior to upgrading.
- Use the Unable to show "metadata-from": No such page "_space_variables"to determine which node within the VNF currently has the standby role, or use the following CLI procedure:
- Log into the CLI of one of the nodes.
- Issue the command:
show status system rgStatus
- Check the output to determine which node shows its
currentRole
asstandby.
Upgrading Deployments that Include OAM Nodes
If the deployment you are upgrading includes OAM nodes, VNFM instantiates the OAM nodes in their own cluster, separate from the redundancy groups containing the SBC nodes the OAM nodes manage. When upgrading the deployment, upgrade the OAM nodes in the VNF first, and then upgrade the SBC nodes. Typically, the OAM nodes are deployed in a 1:1 HA configuration including an active and a standby OAM node. Upgrade the standby OAM node first. After upgrade of the standby OAM completes, manually switchover the active OAM node (request system admin <system name> switchover
) to switch roles. Then, upgrade the new standby (former active) OAM node. Once you complete upgrade of the OAM nodes you can continue with upgrading the SBC nodes.
In N:1 model, the VNF has a pair of OAM nodes and one or more redundancy groups. You can upgrade all these in rolling and automated fashion with no service impact.
Upgrading for the bullseye SBC
The VNFM is primarily tested with an automated upgrade system. With manual upgrades, the user has the control and can choose which nodes can be upgraded. You can select both nodes to upgrade to avoid gluster issues. The VNFM upgrades both nodes and there is service impact for OAM nodes. On a successful OAM node upgrade, you can select the managed node standby first and active. There is no service impact for the managed node. A user must manually log into each node, check the status, and once they are up and running then proceed with upgrade with other nodes.
When an automated option upgrade is chosen first, it prioritizes each node and upgrades each node sequentially. This upgrade is done similarly to a hardware based approach where it starts with the standby OAM and later an active OAM. Once an upgrade is successful, it continues moving to a managed node. There is no service impact on the nodes. The OpenStack approach is different, as the stack is deleted first and the new SBC version is installed. There is a service impact during this upgrade.
On a VNFM automated upgrade, there is an issue when upgrading from a buster to a bullseye SBC version. This occurs in buster with a gluster filesystem, with 5.5 version and in bullseye, it is a 9.2 gluster version. There are many changes that went in to gluster 9.2 and gluster 9.2 and the 5.5 version are not compatible. When the Standby OAM is upgraded with gluster 9.2 and active OAM still runs gluster 5.5, they do not communicate correctly and the nodes must exchange messages to further proceed with an upgrade.
Due to these limitations, an automated VNFM upgrade is not supported for a buster to bullseye upgrade.
Upgrading the VNFM Manually
- Choose the manual upgrade option.
- Select both OAM node and perform an upgrade.
- Both OAM nodes are deleted and re-created.
- Log into each node and once they are up and running, go for other nodes.
- For other non OAM nodes/managed nodes, select the standby and perform an upgrade
- Once the upgrade is complete and the service is up, select another node.
Monitoring Node Status During Upgrade
Before and during the upgrade procedure, use the SBC CLI to check the status of the nodes in the redundancy group (RG) you are upgrading:
- Log into the CLI of one of the nodes.
- Issue the command:
show status system rgStatus
and check the output to:- Determine which node shows its
assignedRole
asstandby
. This is the node to upgrade first. - Verify that the
syncStatus
of all the currently active SBC VM(s) issyncCompleted
. - Verify that the
syncStatus
of the current standby SBC VM isunprotectedRunningStandby
.
- Determine which node shows its
The following is an example of the type of output that would appear in a full 4:1 HA deployment. Smaller deployments would include fewer node entries. In a 1:1 HA configuration, the output would include entries for one active and one standby node, similar to the first two entries in the example.
> show status system rgStatus rgStatus vsbc1-192.168.2.3 { actualCeName vsbc1; assignedRole active; currentRole active; nodeId 1; serviceId 0; syncStatus syncCompleted; usingMetavarsOf vsbc1-192.168.2.3; appVersion V08.00.00; } rgStatus vsbc1-192.168.2.4 { actualCeName vsbc1; assignedRole standby; currentRole standby; nodeId 2; serviceId 1; syncStatus unprotectedRunningStandby; usingMetavarsOf vsbc1-192.168.2.4; appVersion V08.00.00; } rgStatus vsbc1-192.168.2.5 { actualCeName vsbc1; assignedRole active; currentRole active; nodeId 3; serviceId 2; syncStatus syncCompleted; usingMetavarsOf vsbc1-192.168.2.5; appVersion V08.00.00; } rgStatus vsbc1-192.168.2.6 { actualCeName vsbc1; assignedRole active; currentRole active; nodeId 4; serviceId 3; syncStatus syncCompleted; usingMetavarsOf vsbc1-192.168.2.6; appVersion V08.00.00; } rgStatus vsbc1-192.168.2.7 { actualCeName vsbc1; assignedRole active; currentRole active; nodeId 5; serviceId 4; syncStatus syncCompleted; usingMetavarsOf vsbc1-192.168.2.7; appVersion V08.00.00; }
Throughout the upgrade process, maintain a CLI session in one of the VMs to monitor rgStatus
. When it is time to upgrade the VM where you are monitoring rgStatus
, start a CLI session in an upgraded VM to continue monitoring rgStatus
. While upgrading M-SBC deployments it is also helpful to keep track of which node is the leader node for the cluster by issuing the command: show table system loadBalancingService leaderStatus
The command output lists all of the active nodes in the cluster and the leader node is the first node listed, for example:
> show table system loadBalancingService leaderStatus LEADER LEADER ID IPADDRESS STATE -------------------------- 0 192.168.10.6 completed 1 192.168.10.10 completed [ok][2018-11-15 21:52:51]
Upgrade Procedure
The upgrade procedure on VNFM occurs in two phases. First you upgrade the VNF, during which you specify the software version to which you want to upgrade. Then you upgrade the individual VM nodes (VNFCs) to the specified software version.
Upgrading a VNF
Use the following procedure to upgrade a VNF.
Perform the procedure Performing Onboarding Through the UI. You must add a new version of the VNF to the VNF catalog.
- Click VNF Lifecycle to display all VNFs and VNFCs.
Select the VNF you want to upgrade from the VNF Lifecycle panel.
Select Upgrade from the Select Action drop-down menu for the selected VNF.
NoteIn the VNF Lifecycle panel, the status of the VNF you want to upgrade must be Ready. The Upgrade action is only available if the VNF is in the Ready state.
The Upgrade VNF window appears.VNF Ready to upgrade
Upgrade VNF
Provide a reason for the upgrade (optional). This information is included in logs and passes to the VNFs during the upgrade.
Select the upgrade version from the Upgrade Version drop-down menu.
- Click Save.
The VNF status in the VNF Lifecycle panel updates to Upgrading. Continue with the Upgrading a VNFC procedure to upgrade each VM node, one by one.
After you upgrade each VNFC, the VNF status is Upgraded.
Upgrading a VNFC
Before you can upgrade a VNFC, you must complete the "Upgrading a VNF" procedure that specifies the software version. After completing that procedure, the status for the individual nodes (VNFCs) within the VNF appears as Upgrade Required. The Upgrade action only displays if the VNF status is Upgrading and the VNFC status is Upgrade Required.
Use the following procedure to upgrade a VNFC.
- Log into the VNFM UI. Refer to Logging into the VNFM UI.
- Click VNF Lifecycle to display all VNFs and all VNFCs.
- Select the VNF you are upgrading in the VNF Lifecycle panel. The nodes within the selected VNF are listed below in the VNFC Lifecycle panel.
- In the VNFC Lifecycle panel, select the node you previously determined to be the standby node. The standby node must be upgraded first.
- Log into the CLI of the node you are upgrading and issue the command:
sbxstop
Note: Make sure the process to stop the SBC has completed before continuing. Select Upgrade from the Select Action drop-down menu in the VNFC Lifecycle panel for the node you are upgrading.
The Upgrade VNFC window appears.Upgrade VNFC
Select the upgrade option. Note that if the VM has an attached Cinder boot volume, the Rebuild option is not offered, you must use Recreate (reuse).
- Rebuild - upgrades the VM using the upgrade image. The VM UUID will not change. Not available if the VM has an attached Cinder boot volume.
- Recreate (reuse) - re-creates the VM instance using the upgrade image. The VM UUID will change.
Click Upgrade. VNFM initiates the upgrade.
After the VM status changes to "Upgraded" and the node is back in service, determine the next node to upgrade. In a 1:1 HA deployment, this will be the currently active node. In an N:1 HA deployment, select one of the active nodes. Before upgrading an instance, check the
syncStatus
field in the output of thergStatus command
to ensure that all the currently active instances are in thesyncCompleted
state. The standby instance displayssyncStatus
asunprotectedRunningStandby
.Repeat steps 5 through 8 for the active node in a 1:1 HA deployment. In an N:1 HA deployment, repeat steps 5 through 8 for each active node, one by one, until all VMs are upgraded.
The VNF status in the VNF Lifecycle panel should be Upgrade in Progress until the VNFC upgrades are complete. After each node is upgraded, its status changes to Upgraded. After the last node in the VNF is upgraded, the status of the active node(s) changes to Available. The status of the standby node changes to Maintenance for a short time before it comes into service and then it also becomes Available.
The VNF status in the VNF Lifecycle panel then changes to Ready.
Post-Upgrade Verification
To verify that the nodes instances are up and running with the upgraded software version:
- Log into the CLI of one of the nodes.
- Issue the command:
show status system rgStatus
In the output, check that the value for
syncStatus
for the currently active instance issyncCompleted
. The value for the standby instance should appear asunprotectedRunningStandby
.