Pre-upgrade steps:

At least seven days prior to starting the upgrade, accomplish the following tasks.

1. Verify existing software version of SBC

Refer to 

Verifying the Existing Software Version of SBC
.

2. Check Product Release Interoperabilities

If other Ribbon Core elements are integrated with the SBC, ensure they are running a version compatible with the target SBC load according to 

SBC 5000-7000-SWe Interoperability Matrix
.

3. Verify Bulletins and Warnings (WBAs)

Examine all Warnings/Bulletins/Announcements listed in the "Associated Ribbon Announcements" section for the target release 

Release Notes
to verify they are applicable to the upgrade path in the Ribbon support portal. Log on to the
Ribbon Support Portal
 to view WBA details.

In addition to the Warnings/Bulletins/Announcements listed in the Release Notes, log on to the Ribbon support portal and verify all Warnings/Bulletins applicable to the upgrade path.

4. Review Release Notes (RNs)

Review RNs for the SBC configuration requirements to implement prior to the upgrade.

Some examples are SWe HW configuration requirements and memory requirements for SBC 51xx and 52xx systems, depending on target release and username requirements.

5. Prepare Software

  • Review the target RN "Required Software and Firmware Versions" for the required firmware and application software files required for the upgrade.
  • Transfer the SBC application software to the SBC. Refer to the section
    Transfer the Package to Target Servers
    .
  • After transfer is complete, change file permission of the downloaded files to 'rwxrwxrwx' by running this command as user root:
    chmod 777 /opt/sonus/external/sbc-V*
  • Follow 
    Validating MD5Sum with Checksums Calculator
     steps to confirm the downloaded files include the correct checksum, or validate md5sum directly on the SBC servers after the software is transferred.


6. Verify call arrival rate and maximum number of active calls during upgrade

Call arrival rate limits are enforced during upgrade.

Verify the system call arrival rate (systemCongestionCallArrivalRate) using the following CLI command. The maximum call rate allowed for Live Software Upgrade (LSWU) differs between various SBC platforms.

For more information, refer to 

LSWU Performance Metrics
.

Example:

admin@WFDSBC01a> show status system systemCongestionStatus 
systemCongestionStatus entry { 
    systemCongestionMCLevel 0; 
    systemCongestionCPULevel 0; 
    systemCongestionMemLevel 0; 
    systemCongestionCallRateLevel 0; 
    systemCongestionMCDuration 67397; 
    systemCongestionCallArrivalRate 355; 
    systemCongestionCallAcceptRate 100; 
    systemCongestionCallAcceptCount 0; 
    systemCongestionCallEqArrivalRate 355; 
    systemCongestionRegArrivalRate 0; 
    systemCongestionIRTTLevel 0; 
}

7. Check alarms

Ensure that no Critical/Major alarms are present prior to the upgrade, unless they are confirmed as non-impacting.

To check alarms via EMA:
Log in to EMA on the active server, and navigate to: Monitoring -> Dashboard -> Alarms -> Current alarms.

For earlier versions of EMA (4.x):
Login to the EMA on the active server, and navigate to: Troubleshoot -> Alarms -> Current.

To check alarms via CLI:
% show table alarms currentStatus

8. Check disk space requirements

For upgrades to 05.xx.xx:
Refer to

Warning-15-00022084
 Hard disk usage requirements for upgrading to version 05.xx.xx.

For upgrades to 06.xx.xx or higher releases:
Maximum of 70% used and at least 3G free space on '/' partition
Maximum of 80% used and at least 7G free space on '/home' partition
Maximum of 70% used and at least 7G free space on '/dev/drbd0' partition (active node only)

Note

The automated pre-check covers this step.

To check for disk space, run df -k -h -P.

9. Check disk speed requirements

This step is applicable ONLY for hardware SBCs.


To check for disk speed, run hdparm -t --direct rootDisk. Run the check three times and follow the steps in Bulletin-23-000007449 if at least one test returns a disk speed below 100MB/s.

Example: 

[root@example.org ~]# hdparm -t --direct /dev/sda

/dev/sda:

Timing O_DIRECT disk reads: 584 MB in 3.01 seconds = 193.85 MB/sec
[root@example.org ~]#

 To obtain the root disk, perform the following command as root:

cat /proc/partitions 2> /dev/null | awk '$2 == 0 { print $NF }' | grep -Ev "dm-0|sr0|drbd0|fd0"

The rootDisk partition is /dev/ followed by the output of the command above.

SSD absolute minimum speed is 100 MB/s. The upgrade will abort and fail if the SSD speed falls below this threshold. Ribbon recommends that you take action if the disk speed falls below 100MB/s to ensure that the disk speed is sufficient during intervals of higher system resources usage. Follow the steps in in Bulletin-23-000007449.

10. Verify the SBC platform is running in redundant mode (for HA systems)

Note

The automated pre-check covers this step.

Use the "show table system serverStatus" CLI command, and confirm "MGMT REDUNDANCY ROLE" shows active/standby:

Example (shortened for brevity):

admin@LABNBS2A> show table system serverStatus
                                                                         MGMT                                                                         
                                           PLATFORM       APPLICATION    REDUNDANCY                    APPLICATION UP    LAST RESTART                 
NAME      HW TYPE   SERIAL NUM  PART NUM   VERSION        VERSION        ROLE        UP TIME           TIME              REASON         SYNC STATUS   
------------------------------------------------------------------------------------------------------------------------------------------------------
LABNBS2A  SBC 5200  2038100494  821-00430  V06.02.03R000  V06.02.03R000  active      21 Days 21:59:38  21 Days 21:50:18  systemRestart  syncCompleted 
LABNBS2B  SBC 5200  4031110016  821-00430  V06.02.03R000  V06.02.03R000  standby     10 Days 00:14:02  10 Days 00:12:00  systemRestart  syncCompleted 

11. Verify the sync status

Note

The automated pre-check covers this step.

Use the "show table system syncStatus" CLI command.

Example for HA systems:

admin@LABNBS2A> show table system syncStatus
SYNC MODULE             STATUS
---------------------------------------
Policy Data             syncCompleted
Disk Mirroring          syncCompleted
Configuration Data      syncCompleted
Call/Registration Data  syncCompleted

Example for Standalone systems:

admin@LABSTD> show table system syncStatus
SYNC MODULE             STATUS
-------------------------------------
Policy Data             unprotected
Disk Mirroring          unprotected
Configuration Data      unprotected
Call/Registration Data  unprotected

12. Verify that at least one management port is in admnEnabledPortUp state on each server

Use the show table system ethernetPort mgmtPortStatus CLI command:

Example:

admin@LABNBS2A> show table system ethernetPort mgmtPortStatus
          PORT  IF                                            RX        TX       RX      TX      RX       TX
CE NAME   NAME  INDEX  MAC ADDRESS       LINK STATE           PACKETS   PACKETS  ERRORS  ERRORS  DROPPED  DROPPED
-------------------------------------------------------------------------------------------------------------------
LABNBS2A  mgt0  1      0:10:6b:2e:e7:91  admnEnabledPortUp    27189224  2115956  0       0       0        0
LABNBS2A  mgt1  2      0:10:6b:2e:e7:92  admnEnabledPortUp    7474875   45345    0       0       0        0
LABNBS2B  mgt0  3      0:10:6b:2:ee:76   admnEnabledPortUp    34677     354      0       0       0        0
LABNBS2B  mgt1  4      0:10:6b:2:ee:77   admnEnabledPortUp    35362     425      0       0       0        0

13. Verify if the system and the databases are in sync with each other

The steps to accomplish this check are included in 

Warning-14-00020748
.


14. Verify the number of historical alarms

The steps to accomplish this check are included in 

Warning-21-00029972
.

Applicable for the upgrades from pre-9.1 versions only.

15. System uptime check

15.1 For SBC SWe

Check how many days the SBCs are up ('UP TIME') using the following CLI command.

Example (shortened for brevity):

admin@SWE1> show table system serverStatus
                                                                                               MGMT                                         
                                                           PART  PLATFORM       APPLICATION    REDUNDANCY                   APPLICATION UP  
NAME  HW TYPE        SERIAL NUM                            NUM   VERSION        VERSION        ROLE        UP TIME          TIME            
--------------------------------------------------------------------------------------------------------------------------------------------
SWE1  Sonus SBC SWe  15AF4D56-2731-4822-F9AA-BBCCDD62C816  -     V06.01.00R001  V06.01.00R001  active      9 Days 00:14:03  9 Days 00:11:56 
SWE2  Sonus SBC SWe  4F874D56-1403-209B-E224-1D84F5C99A7C  -     V06.01.00R001  V06.01.00R001  standby     9 Days 00:11:09  9 Days 00:07:01 

If SBCs are up for more than 200 days, reboot the servers prior to upgrade. Reboot the standby SBC first (click here for reboot procedure). Then once the SBCs are back in sync again, switchover the SBC application and reboot the new standby SBC.

15.2 For SBC5XX0/SBC7000

  • Check how many days the BMC is up using 'uptime' command on the BMC ssh session.
  • If the BMC is up for more than 90 days, reboot the BMC prior to the upgrade according to the steps below, depending on the BMC version.

15.2.1. SSH to the SBC associated to the standby BMC as root user and ensure is standby:

    sbxstatus | grep 'Service running'
            ** Service running [standby] **

15.2.2. Stop the server on standby SBC, as root user:

    sbxstop

15.2.3. Power off the server via BMC GUI:

Remote Control -> Server Power Control -> Power Off Server - Orderly Shutdown -> Perform Action

15.2.4. Ensure after Orderly Shutdown, server is showing 'Host is currently off' on BMC:

BMC Remote Control -> Server Power Control ("Power Control" in 2.x BMC versions) window

15.2.5. ssh to the BMC and verify that the SBC is powered off by running the following command.

For BMC versions 2.x and higher, enable ssh access in BMC GUI > Configuration > Network ("BMC Network" in 2.x BMC versions):

ipmitool -H127.0.0.1 -Uroot -Psuperuser power status

The output of the above command should read "Chassis Power is off".

If your BMC ssh root password is not default, change the '-P' parameter value in the above command, accordingly.

15.2.6. Reboot the BMC:

For BMC versions 2.x, reboot the BMC via the GUI: Maintenance -> Reboot BMC.

For other BMC versions, run "reboot" command on the BMC ssh session.

15.2.7. The BMC will come back after 2 minutes and the system will boot back to standby mode.

Ensure the following:

- Access to the BMC GUI and remote console is successful (prompt is available)
- SBC is back at standby.
- SBCs are in sync (show table system syncStatus via CLI)

15.2.8. Switchover the SBC application and repeat steps 13B.1 - 13B.7 to the newly standby SBC / BMC.

16. Verify System Name and hostname

Note

The automated pre-check covers this step.

  • System name and hostnames (ceName/peerCeName for HA) can only contain alphanumeric characters, dash, or period, must begin with a letter, and end with an alphanumeric character.
  • Obtain the System name from the sbx.conf file: /opt/sonus/conf/sbx.conf file.
  • For earlier releases, the file location is: /opt/sonus/sbx.conf

Example (shortened for brevity):

[root@LABNBS2A ~]# cat /opt/sonus/conf/sbx.conf
# SBX startup configuration.
#
# Copyright (c) 2010 Sonus Networks, Inc.
# All rights reserved.
#
# Configuration generated by sbxInit.sh:
# Thu Oct 31 15:59:03 CET 2019
role=1             # 1=Active, 2=Standby
systemName=LABNBS2
ceName=LABNBS2A
peerCeName=LABNBS2B

17. Confirm no USB or mounted devices are present on the SBC prior to the upgrade to avoid upgrade issues

18. Run LSWU pre-checks

Note

This step is only applicable for HA setup. You can use it for offline upgrade as well.


Note

LSWU pre-checks using the Upgrade Manager interface do not affect any ongoing SBC services.

Prior to performing LSWU pre-checks using the Upgrade Manager interface, ensure:

To perform LSWU pre-checks:

  1. On the PM or EMA of the active server, navigate to Administration > System Administration > Software Install/Upgrade. The Software Install/Upgrade window is displayed. The available packages are displayed in the Packages panel.
  2. In the Packages panel, select the package that you want to upgrade to.

  3. Click Live Software Upgrade to continue with the LSWU pre-checks. The SBC validates the selected package.

    Once the package is verified, the SBC displays the upgrade option.

    LSWU Pre-checks Window

  4. Click Perform Pre-Upgrade Checks.

    Note:

    You can perform pre-upgrade checks anytime, including outside the maintenance window. These checks verify if the system is ready for an upgrade.

    Pre-Upgrade Checks Complete message is displayed after the verification is complete or error messages are displayed if there are pre-check failures. Ensure to resolve the errors and repeat the pre-upgrade checks to confirm these are successful prior to the actual upgrade start.

    IMPORTANT

    After the pre-checks are run, press Cancel to abort the actual upgrade process.

    LSWU Pre-Checks in Progress