In this section:
Live Software Upgrade (LSWU) allows you to upgrade the SBC Core application in an HA environment without dropping active calls or interrupting service. This upgrade approach is recommended for High Availability SBC systems. For information on average call rates and capacities during an LSWU, refer to LSWU Performance Metrics.
If you are upgrading from any version prior to 7.1.0, you must have no sessions in the /var/log/sonus/evlog/
or /var/log/sonus/evlog/evlog/
directories.
(This Note does not apply if you are upgrading from 6.2.2, 7.1.0 or later versions.)
Verify the SBC platform is running in redundant mode using the following CLI command:
admin@WFDSBC01a> show table system serverStatus MGMT DAUGHTER PLATFORM APPLICATION REDUNDANCY APPLICATION UP LAST RESTART BOARD NAME HW TYPE SERIAL NUM PART NUM VERSION VERSION ROLE UP TIME TIME REASON SYNC STATUS PRESENT ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- WFDSBC01a SBC 5200 2044090074 821-00430 V07.00.00R000 V07.00.00R000 active 6 Days 15:32:47 6 Days 15:31:04 systemRestart syncCompleted true WFDSBC01b SBC 5200 2052090007 821-00430 V07.00.00R000 V07.00.00R000 standby 6 Days 15:32:47 6 Days 15:31:04 systemRestart syncCompleted true
Verify the sync status using the following CLI command.
admin@WFDSBC01a> show table system syncStatus SYNC MODULE STATUS ------------------------------------------- Policy Data syncCompleted Disk Mirroring syncCompleted Configuration Data syncCompleted Call/Registration Data syncCompleted
Verify that at least one management port is in admnEnabledPortUp state on each server using the following CLI command:
admin@WFDSBC01a> show table system ethernetPort mgmtPortStatus CE PORT IF RX TX RX TX RX TX NAME NAME INDEX MAC ADDRESS LINK STATE PACKETS PACKETS ERRORS ERRORS DROPPED DROPPED --------------------------------------------------------------------------------------------------------------------------- WFDSBC01a mgt0 1 0:10:6b:2e:e6:9e admnEnabledPortUp 1728556 132270 0 0 0 0 WFDSBC01a mgt1 2 0:10:6b:2e:e6:9f admnEnabledPortUp 6754292 6480 0 0 0 0 WFDSBC01b mgt0 3 0:10:6b:2e:e5:ea admnEnabledPortUp 3457239 0 0 0 0 0 WFDSBC01b mgt1 4 0:10:6b:2e:e5:eb admnEnabledPortUp 1038353 0 0 0 0 0
Verify the system call arrival rate (systemCongestionCallArrivalRate) using the following CLI command. The maximum call rate allowed for LSWU differs between various SBC platforms. For more information, refer to LSWU Performance Metrics page.
admin@WFDSBC01a> show status system systemCongestionStatus systemCongestionStatus entry { systemCongestionMCLevel 0; systemCongestionCPULevel 0; systemCongestionMemLevel 0; systemCongestionCallRateLevel 0; systemCongestionMCDuration 67397; systemCongestionCallArrivalRate 355; systemCongestionCallAcceptRate 100; systemCongestionCallAcceptCount 0; systemCongestionCallEqArrivalRate 355; systemCongestionRegArrivalRate 0; systemCongestionIRTTLevel 0; } [ok]
Set the log filter levels (in configuration mode) to major using the following CLI command.
admin@WFDSBC01a% set oam eventLog typeAdmin debug filterLevel major admin@WFDSBC01a% set oam eventLog typeAdmin system filterLevel major admin@WFDSBC01a% commit
Delete the LI configuration before performing the LSWU.
Log on as Calea user with Calea credentials.
Check the current LI state using the command:
> show table addressContext default intercept callDataChannel <callDataChannel_Name> interceptStandard packetcable; vendorId none; priState enabled; secState disabled; priMode active; secMode outOfService; priIpAddress 0.0.0.0; secIpAddress 0.0.0.0; priPort 0; kaTimer 5; retries 3; ipInterfaceGroupName IpIntGrp1; [ok]
Delete the LI Configuration using the commands:
Prior to deleting an LI Configuration, set the following flags to:
priMode - outOfService
priState - disabled
secMode - outOfService
secState - disabled
% set addressContext default intercept callDataChannel <callDataChannel_Name> priMode outOfService % commit % set addressContext default intercept callDataChannel <callDataChannel_Name> secMode outOfService % commit % set addressContext default intercept callDataChannel <callDataChannel_Name> priState disabled % commit % set addressContext default intercept callDataChannel <callDataChannel_Name> secState disabled % commit % delete addressContext default intercept callDataChannel <callDataChannel_Name> % commit
Verify if the system and the databases are in sync with each other.
For SBC 5100, SBC 5110, SBC 5200, and SBC 5210 only: Prior to performing a Live Software Upgrade (LSWU), verify if the system and the databases are in sync. The steps to accomplish this are included in the announcement Warning-14-00020748. Refer to the applicable release notes for instructions on viewing announcements.
Before LSWU, ensure that the number of rules across profiles in a system is limited to 2000, and the number of actions across profiles in a system is limited to 10000.
This procedure prepares the server for revert, upgrades OS (if required) and upgrades the SBC application. During this process, the server may reboot up to two times.
At the CLI prompt of active server, start the LSWU using the following CLI command:
admin@WFDSBC01a> request system serverAdmin WFDSBC01b startSoftwareUpgrade package sbc-V07.01.00-R000.x86_64.tar.gz This command will start live software upgrade. Do you want to proceed (yes/no): <yes> Proceeding result success reason Success from: WFDSBC01b
Once the upgrade starts, monitor the status using the following CLI command.
admin@WFDSBC01a> show table system serverSoftwareUpgradeStatus NAME UPGRADE STATUS ---------------------------- WFDSBC01a pendingUpgrade WFDSBC01b upgrading
At the beginning of the upgrade, the server is rebooted and disk partitioning is done automatically which takes approximately 15 to 20 minutes. Do not hit the power button or restart the server during this time. Open the JViewer console to watch the progress of the upgrade.
Verify the sync status is in syncCompleted state, using the following CLI command.
admin@WFDSBC01a> show table system syncStatus SYNC MODULE STATUS ------------------------------------------- Policy Data syncCompleted Disk Mirroring syncCompleted Configuration Data syncCompleted Call/Registration Data syncCompleted
During an LSWU process, when the Standby SBC is in "Sync" state (which usually takes about 2 - 3 minutes), the new call data is queued on the Active SBC. Those calls are not marked as stable. Once the "Sync" is complete, the pending call data queue is moved from Active SBC to Standby SBC. Those calls are marked as stable. Therefore, check the callCountStatus
when both Active and Standby SBCs complete the "Sync" state. IRTT congestion is expected while “Sync” is in progress and should get cleared when Active and Standby SBCs complete the “Sync” state.
Verify the software upgrade status is in Upgraded state, using the following CLI command.
admin@WFDSBC01a> show table system serverSoftwareUpgradeStatus NAME UPGRADE STATUS -------------------------------- WFDSBC01a pendingUpgrade WFDSBC01b upgraded
This process takes approximately 45 minutes to complete on a single server.
Verify the system server status using the following CLI command.
admin@WFDSBC01a> show table system serverStatus MGMT PLATFORM APPLICATION REDUNDANCY APPLICATION UP LAST RESTART NAME HW TYPE SERIAL NUM PART NUM VERSION VERSION ROLE UP TIME TIME REASON SYNC STATUS ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- WFDSBC01a SBC 5200 2044090074 821-00430 V07.00.00R000 V07.00.00R001 active 0 Days 02:44:22 0 Days 02:43:04 systemRestart syncCompleted WFDSBC01b SBC 5200 2052090007 821-00430 V07.01.00R000 V07.01.00R000 standby 0 Days 00:13:02 0 Days 00:07:23 softwareUpgradeOrRevert syncCompleted
Upgrade the next server using the following CLI command.
admin@WFDSBC01a> request system serverAdmin WFDSBC01a startSoftwareUpgrade package sbc-V07.01.00-R000.x86_64.tar.gz This command will start live software upgrade. Do you want to proceed (yes/no): <yes> Proceeding result success reason Success from: WFDSBC01a [ok]
Once the upgrade starts, monitor the status using the following CLI command on the new active server.
admin@WFDSBC01b> show table system serverSoftwareUpgradeStatus NAME UPGRADE STATUS ---------------------------- WFDSBC01a upgrading WFDSBC01b upgraded
Verify whether both the servers are upgraded using the following CLI command.
admin@WFDSBC01b> show table system serverSoftwareUpgradeStatus NAME UPGRADE STATUS ---------------------------- WFDSBC01a upgraded WFDSBC01b upgraded
After both the servers get upgraded and is in sync with its peer, an automatic switchover may happen.
Verify the Live Software Upgrade status is in UpgradeDone state and both primary and secondary upgrade status reflect upgraded state. Use the following CLI command.
admin@WFDSBC01a> show table system softwareUpgradeStatus state upgradeDone; previousState upgrading; upgradeStartTime "Wed Dec 06 05:37:04 2017"; revertStartTime n/a; package /opt/sonus/external/sbc-V07.01.00R000-connexip-os_06.01.00-R000_amd64.qcow2; rpmName /opt/sonus/staging/sbc-V07.01.00-R000.x86_64.rpm; upgradeScript /opt/sonus/staging/sbxUpgrade.pl; revertScript /opt/sonus/sbxRevert.pl; reason successfulCompletion; oldRelease V07.00.00R000; newRelease V07.01.00R000; primaryUpgradeStatus upgraded; secondaryUpgradeStatus upgraded;
Verify the SBC server status using the following CLI command.
admin@WFDSBC01a> show table system serverStatus MGMT PLATFORM APPLICATION REDUNDANCY APPLICATION UP LAST RESTART NAME HW TYPE SERIAL NUM PART NUM VERSION VERSION ROLE UP TIME TIME REASON SYNC STATUS ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- WFDSBC01a SBC 5200 2044090074 821-00430 V07.01.00R000 V07.01.00R000 active 0 Days 15:32:47 0 Days 15:31:04 softwareUpgradeOrRevert syncCompleted WFDSBC01b SBC 5200 2052090007 821-00430 V07.01.00R000 V07.01.00R000 standby 0 Days 15:32:47 0 Days 15:31:04 softwareUpgradeOrRevert syncCompleted
Once the upgrade is successful, clear the browser cache.