In this section:
Upgrade the Software
Start
Log onto the CPU as root that contains the stp_sp2k_21_x_x_nbxxxxxxx_upgrade.sh file.
Change the directory to /var/log.
cd /var/log
NoteAfter you upload the software package from your PC to the SP2000, the system automatically does the following:
- Confirms that the software upgrade package is found
- Checks that there is sufficient memory on the SP2000 for the new software
- Extracts the new software to the SP2000 that you are logged into (/var/log)
- Installs the scripts to all Management and Routing CPU cards
Enter the following command to run the Self-Extraction utility and press ENTER.
./stp_sp2k_21_x_x_nbxxxxxxxx_upgrade.sh
Example[root@TMA15sp2kslot14 log]# ./stp_sp2k_21_x_x_nbxxxxxxxx_upgrade.sh DSC Self-Extraction Utility xx_x_x_xxxxxxxx ----------------------------------------------------- Verify archive integrity... Executing install script... Software Upgrade Preparation ---------------------------- root: UpgradeInstaller Executed upgrade installer xx_x_x_nbxxxxxxxx ---------------------------------------- Executed upgrade installer xx_x_x_nbxxxxxxxx Verify /upgrade ... Partition for /upgrade has 1.5 GB Verify execution point... Verify system state... Verify requisite files in the archive... Is it OK to proceed with preparation of the /upgrade directory? (y/n):
Enter y.
ExamplePreparing /upgrade directory... Expand OS image from archive... Place SW load file... Redirect pre_upgrade_check and upgrade... Place new pre_upgrade_check and upgrade... ---------------------------------------------------------------- To complete the process run the upgrade command. (May optionally run pre_upgrade_check independently beforehand). ---------------------------------------------------------------- complete. [root@TMA15sp2kslot14 log]#
Enter the following at the command line and press ENTER.
upgrade
The upgrade is automatically executed on the system. During this process, the CPU 14 and 24 initialize, discover, and install the software load and perform the upgrade/restore using the backup file.
Example[root@TMA15sp2kslot14 log]# upgrade :::::::::::::::::::::::::::::::: Ribbon DSC :::::::::::::::::::::::::::::::: SP2000 CPU Upgrade _____________________________________________ Type "q" to quit from most prompts or CTRL-C. This process will prepare each CPU for upgrade before rebooting each CPU in order, isolating the reboots to avoid outages. Please run a pre-upgrade check. Type "yes" to execute this check (exiting otherwise):
Enter yes and press ENTER.
ExampleCalling pre-upgrade check... ======== Performing upgrade PRE-checks... ========= --------------------------------------------------- USB device check... --------------------------------------------------- Check USB device status on all the CPU or VM slots reported by cardlist/slotmon SLOT: 14 SLOT: 24 PASSED
Press ENTER.
Example--------------------------------------------------- Ping connectivity check... --------------------------------------------------- Ping all slots which are CPUS or IOs in /opt/cpu_ss7gw/current/data/slot.conf to check card availability Ping all the slots reported by cardlist/slotmon to check packet loss -Slot: 14 Packets dropped: 0% -Slot: 24 Packets dropped: 0% -Slot: 12 Packets dropped: 0% -Slot: 22 Packets dropped: 0% PASSED
Press ENTER.
Example--------------------------------------------------- ish connectivity check... --------------------------------------------------- Perform ish test between all CPUS in /opt/cpu_ss7gw/current/data/slot.conf to check slots interactive communication availability ish test from slot 14 to all slots in system... ish test from slot 14 to slot 14 passed. ish test from slot 14 to slot 24 passed. ish test from slot 24 to all slots in system... ish test from slot 24 to slot 14 passed. ish test from slot 24 to slot 24 passed. PASSED
Press ENTER.
Example--------------------------------------------------- RPM database check... --------------------------------------------------- Check RPM database status on all the CPU or VM slots reported by cardlist/slotmon SLOT: 14 RPM database files backed up to /var/lib/rpm/backup-20200115 RPM database is not corrupt. Verifying RPM database. Cleaning up RPM database is valid. SLOT: 24 RPM database files backed up to /var/lib/rpm/backup-20200115 RPM database is not corrupt. Verifying RPM database. Cleaning up RPM database is valid. PASSED
Press ENTER.
Example--------------------------------------------------- Free space check... --------------------------------------------------- -Available RAM space: 139 MB -Available HDD (/var/log) space: 54425 MB -Available Compact Flash (/opt directory) space: 1575 MB PASSED
Press ENTER.
ExampleChecking current Clock Settings... --------------------------------------------------- Current Clock Settings... --------------------------------------------------- TIMEZONE: CPU Slot 14: -05 (EST) CPU Slot 24: -05 (EST) IO Slot 12: -05 (EST) IO Slot 22: -05 (EST) HWCLOCK: CPU Slot 14: Wed 15 Jan 2020 11:51:06 AM EST -0.627910 seconds CPU Slot 24: Wed 15 Jan 2020 11:50:35 AM EST -0.375621 seconds IO Slot 12: IO Slot 22: WALL CLOCK: CPU Slot 14: Wed Jan 15 11:51:08 EST 2020 CPU Slot 24: Wed Jan 15 11:51:08 EST 2020 IO Slot 12: Wed Jan 15 11:51:09 EST 2020 IO Slot 22: Wed Jan 15 11:51:09 EST 2020
Press ENTER.
Example---------------------------------------------------------------- Checking if IO netswitch is working... ---------------------------------------------------------------- PASSED
Press ENTER.
Example--------------------------------------------------- Check Kernel and Mount Points. --------------------------------------------------- No tests to perform.
Press ENTER.
Example--------------------------------------------------- Dumping slot status and watchdog processes... --------------------------------------------------- Dumping CPU status info... AVAILABLE SOFTWARE VERSIONS -------------------------------------------------------------------------------- No new load detected on system. Upload and run load .sh file on /var/log before attempting upgrade. CPU (current) stp_sp2k_21_x_x_nbxxxxxxxx OS: Ribbon7.4.6 (Servicepack) [none applied] CPU (previous) Not_Available OS: Ribbon7.4.18 (Servicepack) RUNNING SOFTWARE VERSIONS (*=update required, ?=update unknown) CARD BUILD SERVICE O/S -SLOT----TYPE----------RELEASE--VERSION----PACK---IMAGE-------CARD STATUS--- -------------------------------------------------------------------------------- Slot 14 MGMT/ROUTING 21.x.x (nbxxxxxxxx) SP: Ribbon7.4.6 24 procs running Slot 24 MGMT/ROUTING 21.x.x (nbxxxxxxxx) SP: Ribbon7.4.6 24 procs running (p=servicepack is partially applied) Dumping IO status info... AVAILABLE SOFTWARE VERSIONS -------------------------------------------------------------------------------- IO (current) AMC308 stp_sp2k_21_x_x_nbxxxxxxxxx OS: rel_12_00_p04.sp2k.img (Servicepack) [none applied] RUNNING SOFTWARE VERSIONS (*=update required, ?=update unknown) CARD BUILD SERVICE O/S -SLOT----TYPE----------RELEASE-VERSION------PACK-----IMAGE------- -------------------------------------------------------------------------------- Slot 12 IO AMC308 21.x.x (nbxxxxxxxx) SP: OS: rel_12_00_p04.sp2k.img Slot 22 IO AMC308 21.x.x (nbxxxxxxxx) SP: OS: rel_12_00_p04.sp2k.img (p=servicepack is partially applied) Dumping Watchdog process info for each CPU...
ExampleWatchdog Process Details on MGMT - slot14_0. WID PID STA RPT RHT HTV HTT COM 1 6491 S+ 0 1 10 10 slotmon --platform SP2K -s 14 -g 0 -p /op... 2 16651 S+ 0 0 10 10 pxbr -p /opt/cpu_ss7gw/current/data --pla... 3 16663 S+ 0 0 10 10 upp -p /opt/cpu_ss7gw/current/data/ --pla... 4 16677 S+ 0 0 10 10 sca -p /opt/cpu_ss7gw/current/data/ 5 16707 S+ 0 0 10 10 sysmon.default -p /opt/cpu_ss7gw/current/... 6 16855 S+ 0 0 10 10 sysmon.cpucheck -p /opt/cpu_ss7gw/current... 7 16966 S+ 0 0 10 10 sysmon.diskmon -p /opt/cpu_ss7gw/current/... 8 17136 S+ 0 0 10 10 sysmon.logcpu -p /opt/cpu_ss7gw/current/s... 9 17322 S+ 0 0 10 10 sysmon.logmem -p /opt/cpu_ss7gw/current/s... 10 17474 S+ 0 0 10 10 sysmon.usbcheck -p /opt/cpu_ss7gw/current... 11 17605 S+ 0 0 10 10 sysmon.tipcdump -p /opt/cpu_ss7gw/current... 12 17783 S+ 0 0 10 10 sysmon.slotinventory -p /opt/cpu_ss7gw/cu... 13 17971 S+ 0 0 10 10 sysmon.configuremtu -p /opt/cpu_ss7gw/cur... 14 18145 S+ 0 0 10 10 hwmon --platform SP2K -c 10 15 15308 S+ 0 0 10 10 licensed -r -f /opt/cpu_ss7gw/current/dat... 16 12019 S+ 0 1 10 10 snm --platform SP2K -p /opt/cpu_ss7gw/cur... 17 15331 S+ 0 0 10 10 imfProbe -p 7377 18 17440 S+ 0 5 10 10 gws --platform SP2K -p /opt/cpu_ss7gw/cur... 19 15722 S+ 0 5 10 10 dre --platform SP2K -p /opt/cpu_ss7gw/cur... 20 17439 S+ 0 2 10 10 sccp --platform SP2K --noGTT -p /opt/cpu_... 21 14281 S+ 0 3 10 10 gtt --platform SP2K -p /opt/cpu_ss7gw/cur... 22 17438 S+ 0 5 10 10 dinamo --platform SP2K -p /opt/cpu_ss7gw/... 23 13157 S+ 0 3 10 10 pce --platform SP2K --noGTT -p /opt/cpu_s... 24 15677 S+ 0 0 10 10 imf --platform SP2K -p /opt/cpu_ss7gw/cur...
ExampleWatchdog Process Details on MGMT - slot24_0. WID PID STA RPT RHT HTV HTT COM 1 15063 S+ 0 0 10 10 slotmon --platform SP2K -s 24 -g 0 -p /op... 2 18587 S+ 0 0 10 10 pxbr -p /opt/cpu_ss7gw/current/data --pla... 3 18830 S+ 0 0 10 10 upp -p /opt/cpu_ss7gw/current/data/ --pla... 4 19056 S+ 0 0 10 10 sca -p /opt/cpu_ss7gw/current/data/ 5 19293 S+ 0 0 10 10 sysmon.default -p /opt/cpu_ss7gw/current/... 6 19566 S+ 0 0 10 10 sysmon.cpucheck -p /opt/cpu_ss7gw/current... 7 19810 S+ 0 0 10 10 sysmon.diskmon -p /opt/cpu_ss7gw/current/... 8 20032 S+ 0 0 10 10 sysmon.logcpu -p /opt/cpu_ss7gw/current/s... 9 20289 S+ 0 0 10 10 sysmon.logmem -p /opt/cpu_ss7gw/current/s... 10 20529 S+ 0 0 10 10 sysmon.usbcheck -p /opt/cpu_ss7gw/current... 11 20775 S+ 0 0 10 10 sysmon.tipcdump -p /opt/cpu_ss7gw/current... 12 21038 S+ 0 0 10 10 sysmon.slotinventory -p /opt/cpu_ss7gw/cu... 13 21293 S+ 0 0 10 10 sysmon.configuremtu -p /opt/cpu_ss7gw/cur... 14 21542 S+ 0 0 10 10 hwmon --platform SP2K -c 20 15 20305 R+ 0 0 10 10 licensed -r -f /opt/cpu_ss7gw/current/dat... 16 20323 S+ 0 0 10 10 snm --platform SP2K -p /opt/cpu_ss7gw/cur... 17 20389 S+ 0 0 10 10 imfProbe -p 7377 18 20403 S+ 0 0 10 10 gws --platform SP2K -p /opt/cpu_ss7gw/cur... 19 20784 S+ 0 0 10 10 dre --platform SP2K -p /opt/cpu_ss7gw/cur... 20 20801 S+ 0 0 10 10 sccp --platform SP2K --noGTT -p /opt/cpu_... 21 20812 S+ 0 0 10 10 gtt --platform SP2K -p /opt/cpu_ss7gw/cur... 22 20829 S+ 0 0 10 10 dinamo --platform SP2K -p /opt/cpu_ss7gw/... 23 20841 R+ 0 0 10 10 pce --platform SP2K --noGTT -p /opt/cpu_s... 24 20856 S+ 0 0 10 10 imf --platform SP2K -p /opt/cpu_ss7gw/cur...
Press ENTER.
Example--------------------------------------------------- PCE Specific check for SCCP Parameter... --------------------------------------------------- PASSED
Press ENTER.
Example--------------------------------------------------- GWS Incoming Linkset check --------------------------------------------------- No GWS Incoming Linkset check needed for this SP2K version
Press Enter
Example------------------------------------------------------------------------- Checking if DB Definition of GTT NA and its LPC registration match ------------------------------------------------------------------------- PASSED
Press ENTER.
Example--------------------------------------------------- Audit files check... --------------------------------------------------- 0 files were out of sync. Files: 110 Copied: 0 Deleted: 0 PASSED
Press ENTER.
Example--------------------------------------------------- Reminder to take a backup... --------------------------------------------------- -It is highly recommended to take a backup of the system prior to starting the upgrade. Label and keep the backup in a safe place.
Press ENTER.
Example====== The results of the PRE upgrade check are stored in /var/log/cpu_ss7gw/upgrade_data/PRE_UPGRADE_DATA.stp_sp2k_xx_x_x_nbxxxxxxxx.Wed_Jan_15_11_49_28_EST_2020 ====== Upgrade PRE checks completed. All checks PASSED. You may now proceed with the upgrade. Collecting information from the system. Please wait... Getting software versions [ ==================== ] 100% Using: /shared/log/stp_sp2k_xx_x_x_nbxxxxxxxx_upgrade.sh Upgrading to: 21_x_x_nbxxxxxxxx Requires update: 24 14 Up to date: none I/O Cards: 12 22 Uncommissioned/Bad: Confirmation to proceed with upgrade operations? (yes/no):
Enter yes and press ENTER.
ExampleCopying upgrade package and preparing each CPU. Working on slot: 24 Execute /var/log/stp_sp2k_21_x_x_nbxxxxxxxx_upgrade.sh on 24 Issue backup on localhost Copy backup to peers [24, 14]. Should user input (a y/n prompt) be required after each CPU is upgraded? (yes/no) [n]:
Enter yes and press ENTER.
ExampleUpgrade execution order [24, 14]. To continue, press the Enter key:
Press ENTER.
ExampleProcessing peer 24... Boot Update ----------- Available kernel version: 3.10.0-693.21.1.el7.sonus07.x86_64 Existing kernel version: 3.10.0-693.21.1.el7.sonus07.x86_64 Installed kernel version(s): 3.10.0-693.21.1.el7.sonus07.x86_64 Removing kernel file collection from defunct-upgrade location Removing defunct-upgrade grub config file Removing defunct-upgrade grub2 config file Moving current kernel file collection to rollback location Installing new kernel file collection: file installed: vmlinuz-3.10.0-693.21.1.el7.sonus07.x86_64 file installed: System.map-3.10.0-693.21.1.el7.sonus07.x86_64 file installed: config-3.10.0-693.21.1.el7.sonus07.x86_64 file installed: initramfs-3.10.0-693.21.1.el7.sonus07.x86_64.img file installed: symvers-3.10.0-693.21.1.el7.sonus07.x86_64.gz file installed: vmlinuz-0-rescue-29512d3199624fc5958c44c518e35267 file installed: initramfs-0-rescue-29512d3199624fc5958c44c518e35267.img Update mounts old backup is removed /upgrade/etc/.fstab_previous_0 generated backup /upgrade/etc/fstab_previous_0 set new load to FS2 Generate grub updates. old backup is removed /boot/grub/.grub.conf_previous_3.10.0-693.21.1.el7.sonus07.x86_64 generated backup /boot/grub/grub.conf_previous_3.10.0-693.21.1.el7.sonus07.x86_64 old backup is removed /boot/grub2/.grub.cfg_previous_3.10.0-693.21.1.el7.sonus07.x86_64 generated backup /boot/grub2/grub.cfg_previous_3.10.0-693.21.1.el7.sonus07.x86_64 Clean /upgrade over-mount directories. Preserve/set cloud-init disable Commit boot service Ready to reboot. Waiting for the new machine 24 to recover.
WarningWhen the following "Unable to determine" error occurs, DO NOT interrupt the upgrade process on the current machine. This message is expected. DO NOT type "yes" until verified that this process is complete or a system failure will occur (this is continued in the next section on this page).
- Open a second terminal window for your second CPU machine.
Verify that the second machine is rebooted and running.
TipIf the prompt shows "dscbuild" for the second machine, the update is complete. When the machine is up and running, host name and slot number prompt will replace "dscbuild".
Type wd on the CPU that was just upgraded and ensure that processes are running without issues.
Example[root@SP2K00slot24 ~]# wd WID PID STA RPT RHT HTV HTT COM 1 15860 S+ 0 0 10 10 slotmon --platform SP2K -s 24 -g 0 -p /op... 2 19096 S+ 0 0 10 10 pxbr -p /opt/cpu_ss7gw/current/data --pla... 3 19114 S+ 0 0 10 10 upp -p /opt/cpu_ss7gw/current/data/ --pla... 4 19125 S+ 0 0 10 10 licensed -r -f /opt/cpu_ss7gw/current/dat... 5 19131 S+ 0 0 10 10 sca -p /opt/cpu_ss7gw/current/data/ 6 19193 S+ 0 0 10 10 snm --platform SP2K -p /opt/cpu_ss7gw/cur... 7 19212 S+ 0 0 10 10 sysmon.default -p /opt/cpu_ss7gw/current/... 8 19315 S+ 0 0 10 10 gws --platform SP2K -p /opt/cpu_ss7gw/cur... 9 19335 S+ 0 0 10 10 sysmon.cpucheck -p /opt/cpu_ss7gw/current... 10 19460 S+ 0 0 10 10 sysmon.diskmon -p /opt/cpu_ss7gw/current/... 11 19637 S+ 0 0 10 10 sysmon.logcpu -p /opt/cpu_ss7gw/current/s... 12 19812 S+ 0 0 10 10 sysmon.logmem -p /opt/cpu_ss7gw/current/s... 13 20028 S+ 0 0 10 10 sysmon.usbcheck -p /opt/cpu_ss7gw/current... 14 20216 S+ 0 0 10 10 dre --platform SP2K -p /opt/cpu_ss7gw/cur... 15 20243 S+ 0 0 10 10 sysmon.slotinventory -p /opt/cpu_ss7gw/cu... 16 20335 S+ 0 0 10 10 dinamo --platform SP2K -p /opt/cpu_ss7gw/... 17 20367 S+ 0 0 10 10 sysmon.configuremtu -p /opt/cpu_ss7gw/cur... 18 20528 S+ 0 0 10 10 sysmon.configaudit -p /opt/cpu_ss7gw/curr... 19 15509 S+ 1 0 10 10 hwmon --platform SP2K -c 20 [root@SP2K00slot24 ~]#
- In the GUI, on the CPU slot that has been upgraded, compare your system's status to the state it was in before upgrading (refer to Verify and Record System Status).
- Check Alarms to ensure no unexpected logs or alarms have been flagged.
- Verify MTP3 SS7 Link Status by going to the SP2000 GUI > Applications > MTP3 > All Links. All SS7 links should be In-Service.
- Verify SG M3UA ASP Path Status by going to the SP2000 Web GUI > Applications > SG > M3UA > M3UA Multi ASPs. All M3UA ASP Associations should be “UP”.
- Monitor C20 core alarm dashboard
- Wait 10 minutes for applications and links to stabilize.
Proceed to stop the software on the other (non upgraded) CPU if you wish to perform some extra tests. When the upgrade process of the first CPU has been completed, tests can be performed to ensure stability and functionality of the new load.
Ensure Test Calls are Processed by the New Software
Start
- In the GUI, stop the SS7 Routing Software Applications on the CPU slot that has not been upgraded.
Log into the WEB UI on the CPU that has not been upgraded and navigate to the System Start/Stop Commands Under System > Processes.
Choose the CPU Slot that has not been upgraded and select Stop Routing Software Applications. Click ‘Continue’.
Proceed with test calls.
In the terminal window from the previous section (Step 23), continue to upgrade the second CPU by typing yes.
yes
CautionIf the screen has timed out, and shows a black screen, do not attempt to "wake" the window- just type "yes".
Press ENTER.
ExampleProcessing local 14... Boot Update ----------- Available kernel version: 3.10.0-693.21.1.el7.sonus07.x86_64 Existing kernel version: 3.10.0-693.21.1.el7.sonus07.x86_64 Installed kernel version(s): 3.10.0-693.21.1.el7.sonus07.x86_64 Removing kernel file collection from defunct-upgrade location Removing defunct-upgrade grub config file Removing defunct-upgrade grub2 config file Moving current kernel file collection to rollback location Installing new kernel file collection: file installed: vmlinuz-3.10.0-693.21.1.el7.sonus07.x86_64 file installed: System.map-3.10.0-693.21.1.el7.sonus07.x86_64 file installed: config-3.10.0-693.21.1.el7.sonus07.x86_64 file installed: initramfs-3.10.0-693.21.1.el7.sonus07.x86_64.img file installed: symvers-3.10.0-693.21.1.el7.sonus07.x86_64.gz file installed: vmlinuz-0-rescue-29512d3199624fc5958c44c518e35267 file installed: initramfs-0-rescue-29512d3199624fc5958c44c518e35267.img Update mounts old backup is removed /upgrade/etc/.fstab_previous_0 generated backup /upgrade/etc/fstab_previous_0 set new load to FS1 Generate grub updates. old backup is removed /boot/grub/.grub.conf_previous_3.10.0-693.21.1.el7.sonus07.x86_64 generated backup /boot/grub/grub.conf_previous_3.10.0-693.21.1.el7.sonus07.x86_64 old backup is removed /boot/grub2/.grub.cfg_previous_3.10.0-693.21.1.el7.sonus07.x86_64 generated backup /boot/grub2/grub.cfg_previous_3.10.0-693.21.1.el7.sonus07.x86_64 Clean /upgrade over-mount directories. Preserve/set cloud-init disable Commit boot service Ready to reboot. You have new mail in /var/spool/mail/root [root@TMA15sp2kslot14 log]# Connection closing...Socket close.
- If testing is unfavorable after the first CPU upgrade AND a rollback is required, apply the following steps BEFORE proceeding with the Rollback. This will minimize the outage on the M3UA ASPs to the CORE.
If there are any issues or unexpected errors with the upgraded CPU, it is recommended that you contact Customer Support.
Verify Software is Properly Installed on the CPUs
Start
- At the Linux prompt, enter startmenu and press ENTER.
- Select System and press ENTER.
- Navigate to the Software Revisions screen.
- Select CPU Versions and press ENTER.
Make sure that the SS7 software and Linux software has been upgraded, all cards are running the same software.
The output of the CPU Versions shows that the CPU card has “UPGRADED” status if this is the first time the command is issued after an upgrade. All subsequent CPU Versions selections show the number of processes running on each CPU.
- Press ENTER.