SBC Core 08.02.03R001 Release Notes

Portfix SBX-105351: The SCM Process cored - UasNotifyRequestCmd.

Impact: The SCM cores after a long extensive load of OOD relay.

Root Cause: The SBC accesses an invalid pointer of link list when the cleanup relay control block.

Steps to Replicate: The system required HW_TYPE_SBS5000 and max support Relay per SCM is 4194304.
Run the OOD relay load (Notify). After a long period time (at least 4m relay per SCM).

The code is modified properly to initialize the link list.

Workaround: None.

Portfix SBX-105156: The SBC was sending m=application 0 UDP/BFCP (NULL).

Impact: The SBC generates the syntax error m line UDP/BFCP in 183.

Root Cause: The SBC accesses the initialize data when the format m-line for UDP/BFCP.

Steps to Replicate: Incoming call with m-line UDP/BFCP. ERE/PSX using script NO_ROUTE and trigger announcement. The internal 183 has m-line UDP/BFCP with invalid syntax.

Make an m-line initialize properly.

Workaround: Use SMM to correct m-line syntax error.

Portfix SBX-104484: The SBC is rejecting the second Update message from the Egress as a 500 result into call failure.

Impact: The SBC rejects a second Update from the Egress due to the DLRBT feature.

Root Cause: The first Update SIPS responds with a locally 200OK but not clear the server request message. As result of the second Update, trigger the SIPS answer 500.

Steps to Replicate: Configure the DLRBT, Egress response 183 with the SDP, first Update, second Update.

After a response 200OK, SIPS clears the server request so it can handle the subsequent one.

Workaround: Disable the DLRB.

Portfix SBX-104769: The SCM Process cored while testing the SBX H323 Conformance and Interworking.

Impact: The SCM Process cored while running the SIP-H323 interworking call to verify rfc2833 to inband DTMF.

Root Cause: Accessing of a NULL pointer caused the core dump.

Steps to Replicate: Run a SIP-H323 interworking call with SIP side rfc2833 and H323 side inband DTMF being enabled.

The code is modified to check for a NULL pointer before accessing it.

Workaround: None.

Portfix SBX-105072: The password padding with random characters by the SBC causes the RADIUS server to reject the password.

Impact: The radius password sent to the server has no zero characters at the end, following the password and a NULL.

Root Cause: The radius passwords are padded to 16 characters. The existing implementation did not set those padded characters to 0.

Steps to Replicate:

1. Configure a radius authentication server.
2. Use a password that is less than 15 characters long for the external radius user.
3. Set externalAuthententication to true.
4. Run a tshark session.
5. Login to the CLI.
6. Stop the tshark.
7. View the radius password element in wireshark after configuring the shared secret in wireshark under protocol preferences.

The code is modified so the padded characters are now set to 0.

Workaround: Use 15 or 16 character passwords.

The ICE not working after an upgrade.

Impact: When a call with ICE changes from direct media to passthru, ICE learning does not get enabled on the call.

Root Cause: The code was not re-enabling the ICE FSM to start ICE learning when changing from direct media to passthru.

Steps to Replicate: 

Test 1
--------

1. Establish a call with ICE on ingress and xdmi on egress.
2. Send re-invite from egress without xdmi and complete the re-invite related signaling.
3. Send stuns to ingress media to complete ICE learning and verify call state changes to established and media can be exchanged between ingress and egress.

The code is modified to re-enable the ICE FSM when changing from direct media to passthru to allow the receipt and processing of stun messages to complete ICE learning.

Workaround: N/A

Portfix SBX-103704: The RTP sourceAddressFiltering is not working (call leg dependency).

Impact: Source address validation is not done if the call flow does not involve NAPT learning. For calls that want source address validation but does not have NAPT enabled, the SBC would not validate the source address and end up forwarding the RTP packet to the other endpoint.

Root Cause: There is no way for application to inform Network Processor to validate source if NAPT learning is not enabled.

Steps to Replicate: Involves a CISCO MOH server. 

Allow application to inform Network Processor to validate source even when the NAPT learning is not enabled.

Workaround: Enable the NAPT learning for the call.

The SBC generates the same ICID for more than 1 call, porting the fix for SBX-101887 into 8.2.3F1 did not help.

Impact: Duplicate the ICID generated for two different calls.

Root Cause: The instance specific value is set to 0 every other call, causing the ICID generation to always use 0 in all instances. This results in the same ICID being generated in more than one SCM instance when the calls land within the same microsecond.

Steps to Replicate: Configure the SBC to generate ICID on the egress side. Run a call load and check for duplicate ICIDs.

Always use the correct instance specific value to manipulate the last octet of the MAC address used in ICID generation to address the issue. 

Workaround: N/A

The MGT0 cannot reach the GW.

Impact: The mgt0 interface appears to be inactive. You cannot ping the gateway from SBC-5400.
The ethtool -S mgt0 shows the tx_pause count rising rapidly.

Root Cause: This problem is specific to SBC-5400.

The SBC-5400 has a backpressure mechanism for management ports so that when the receive buffer exceeds a certain level, pause packets are sent out the interface to ask the switch/router to pause transmission momentarily.

The receive buffer count is incremented by a hardware module and decremented by software.
The notification sent from Network Processor to application about sRTP Rollover Counter (ROC) reset is specifying the wrong interface so software decrements the wrong receive buffer count.
This causes the receive buffer count to wrap around resulting in a high value. This in turn triggers the back-pressure mechanism and continuously sends out pause packets.

Steps to Replicate: 

1. Set up a pass-thru SRTP call.
2. The call should be a long duration call so that the Rollover Counter (ROC) in the sRTP packet becomes non-zero.
3. Modify the call so that the sRTP packet's SSRC changes.

Update:

1. Do a long duration sRTP call (about 30 minutes.)
2. Hold the call.
3. Unhold the call.
4. This causes SSRC change and triggers the problem.

Set the correct interface information in the notification to address the issue.

Workaround: 

Run the check_mgt0.py.txt (rename it to check_mgt0.py) python script (contact your designated Ribbon support representative if you need this) from a Linux command.

Ribbon recommends that you add it as a cron job and run it every 20-30 minutes.

How to add cronjob and run it every 20 minutes:

copy check_mgt0.py to /opt/sonus/sbx/scripts

As root in Linux:

crontab -e
*/20 * * * * /opt/sonus/sbx/scripts/check_mgt0.py

The RTT-TTY support verify call flow in the SBC.

Impact: Lower case characters from the t140 packet were not getting converted correctly to Baudot (tty). Also, some other characters were not getting converted as per V.18 A2 table.

Root Cause: The translation table from T140 to Baudot(TTY) only accounted for subset of characters that were present in Baudot (TTY) character set.

Steps to Replicate: 

1. Setup a T140=>Baudot (AMRNB=>G711) call.
2. Create t140 pcap file with all UTF8 characters (128) and validate baudot generation.
3. Create a t140 pcap file with multi-byte sequence and validate baudot.

The code is modified for:

• The t140=>Baudot translation table as per V.18 Annx2.
• Handling valid multi-byte (2,3 and 4 bytes).
• Handling of Byte Order Mark (BOM).
• Invalid byte sequence and substitute valid but unknown multi-byte seq with an apostrophe.

Workaround: Instead of lower case letters, use upper case letters.

Portfix SBX-101887: The SBC was generating the duplicate ICID for different calls.

Impact: Same ICID value generated for two different calls.

Root Cause: The clock resolution used was 10ms leading to using the same timestamp when the subsequent calls were received within 10ms.

Steps to Replicate: Run a load with CDR enabled. Check no two calls have the same ICID.

Use a high resolution time tick of a 1 micro second and add counter for achieving granularity of 100ns to address the issue.

Workaround: N/A

The SCM process coredumps SipSgHashLookup on the OOD OPTIONS/Subscribe.

Impact: The SCM process may coredump while doing a SipSgHashLookup on a OOD OPTIONS/Subscribe message.

Root Cause: The sipsgRelayCbHashTbl was corrupted where a list head is corrupted and there is not additional information to state what caused the corruption. The issue may have been caused by an entry added to the hash table twice (maybe with different keys).

Steps to Replicate: Not reproducible in the lab. 

The code is modified to ensure the hash table entry is explicitly removed from the hash table, even if the hashlookup fails and also avoids adding a duplicate entry.

Workaround: N/A

Portfix SBX-737: The support for the 3072 bit long RSA keys is missing.

Impact: The SBC needs to support the 3072-bit RSA keys in certificates.

Root Cause: A feature change was required to support the 3072-bit RSA keys in certificates in addition to the 2048-bit and 4096-bit RSA keys.

Steps to Replicate: 

1. Test 3072-bit RSA key certificate on client. Make SIP-TLS call from the client to the SBC acting as TLS server.
2. Configure remote certificates with a 3072-bit RSA key.
3. Configure local certificates with a 3072-bit RSA key.
4. Test a 3072-bit RSA key certificate on the SBC acting as SIP-TLS server.

The code is modified for the 3072-bit RSA keys in certificates.

Workaround: None. 

Portfix SBX-101887 (Originated in Release 8.1.0): The SBC was generating a duplicate ICID for different calls.

Impact: The same ICID value generated for two different calls.

Root Cause: The clock resolution used was 10ms leading to using the same timestamp when subsequent calls were received within 10ms.

Steps to Replicate: Run load with CDR enabled. Check no two calls have the same ICID.

Use high resolution time tick of a 1 micro second and add counter for achieving granularity of 100ns.

Workaround: None.

Portfix SBX-101057 (Originated in Release 7.2.5): The iceSupport and SMM iptgStore coredumped.

Impact: The iceSupport and SMM iptgStore configuration, and an invalid incoming call may caused the SBC to core.

Root Cause: After a response failure to the invalid call, the SBC still tries to initiate setup call. As a result, it accesses the invalid address.

Steps to Replicate: Configure the iceSupport and SMM storeIPTG, incoming call with invalid candidate (no UDP).

The code is modified to not try to initiate setup call if the call fails.

Workaround: Use the SMM to drop an incoming call.

There was a switchover after the TLS certificates were renewed.

Impact: When certificate is modified, an old certificate is freed and new certificate is updated. But after freeing the old certificate, the pointer is not set to NULL that can cause undefined behavior due to an invalid access.

Root Cause: The issue is when the customer modified the certificates through the CLI, it resulted in the SAM process to core. This is due to memory corruption.

Steps to Replicate: Changing the TLS certificates when the TLS load is running will result in memory corruption and the core can occur any time if there is active TLS load running and if the existing TLS calls try to access the certificates that were deleted already.

Set the pointer to NULL so that there is not invalid access to the pointer.

Workaround: N/A

The SBC SWe was not generating a switch over trap-sonusCpRedundGroupSwitchOverNotification.

Impact: A switchover notification trap may not be generated.

Root Cause: The event, used to generate the trap, is incorrectly dropped due to the logic that thinks the event is stale.

Steps to Replicate: Perform switchover and ensure the trap is properly generated.

The code is modified to improve the event handling so that the necessary events process and do not mistakenly drop.

Workaround: The problem only occurs during slow processing. Avoid anything that can slow processing down like info level logging, or any external influences on the disk.

There was an issue with the SBC 7000 TCP window size. 

Impact: Add the CLI support to set kernel parameter net.ipv4.tcp_window_scaling

Root Cause: In certain SBC deployment scenarios (a large number of devices running SIP-TLS), tcp_window_scaling caused small TCP window size.

In such cases, the customer had to use Linux command to disable tcp_window_scaling on each SBC node.

Steps to Replicate: 

(1). Show initial setting for tcp_window_scaling on both active (SBCSWE01a) and standby (SBCSWE01b)
[root@SBCSWE01a ~]# cat /proc/sys/net/ipv4/tcp_window_scaling
1
[root@SBCSWE01b ~]# cat /proc/sys/net/ipv4/tcp_window_scaling
1
[root@SBCSWE01b ~]#

(2). Use CLI to change the kernel param value and display the changed value on both active and standby.
admin@SBCSWE01a% set system admin SBCSWE01 kernelParams tcpWindowScaling disable
admin@SBCSWE01a% commit
Commit complete.

[root@SBCSWE01a ~]# cat /proc/sys/net/ipv4/tcp_window_scaling
0
[root@SBCSWE01a ~]#

[root@SBCSWE01b ~]# cat /proc/sys/net/ipv4/tcp_window_scaling
0
[root@SBCSWE01b ~]#

The code is modified to fix the issue and the Linux command is no longer needed with the modified code.

Workaround: Use the Linux command to set the kernel parameter on both active and standby.

[root@SBCSWE01a ~]# sysctl net.ipv4.tcp_window_scaling
1
[root@SBCSWE01a ~]# sysctl -w net.ipv4.tcp_window_scaling=0
[root@SBCSWE01b ~]# sysctl net.ipv4.tcp_window_scaling
1
[root@SBCSWE01b ~]# sysctl -w net.ipv4.tcp_window_scaling=0

To be persistent across the SBC restart and system reboot, use the Linux the following shell command on both active and standby.
[root@SBCSWE01a ~]# echo "net.ipv4.tcp_window_scaling=0" >> /etc/sysctl.conf
[root@SBCSWE01b ~]# echo "net.ipv4.tcp_window_scaling=0" >> /etc/sysctl.conf

There was a switchover with the PRS process coredumping.

Impact: The PRS process coredumped due to memory corruption.

Root Cause: Customer encountered a PRS process core due to the memory corruption that happened earlier in processing. The core is the after effect of earlier corruption.

Steps to Replicate: None.

The code is modified to avoid the core, and to collect more diagnostic data if a core happens again.

Workaround: N/A

Portfix SBX-100779 (Originated in Release 7.2.5): The SBC was adding an extra application/ISUP ACM content.

Impact: The 200 OK Bye has duplicated the ISUP body.

Root Cause: When the e2e Bye is enabled, 200 OK has duplicated the ISUP body.

Steps to Replicate:

1. Configure the isupMimeBodyRelay and e2e bye.
2. Run a sipi-sipi call, and the SBC sends 200 OK bye has duplicated isup body.

The code is modified so the SBC does not create an isup body internally for the e2e Bye and isupMimeBodyRelay.

Workaround: Disable the e2e Bye.

There was a back-to-back coredump on the Server.

Impact: The SCM process cored due to memory corruption while parsing an INVITE with a very long and invalid CallId field, while the parseError trace logging was enabled.

Root Cause: The SCM process coredump is caused by memory corruption that is the result of the Trace Logging code overwriting a buffer.

This can happen when the SIP parser encounters an invalid CallId that is also very long.
It is also possible that other parsing errors can trigger this core, especially if the parse error was encountered for a particular header that is very long.

Steps to Replicate: This can be reproduced by enabling the parseError trace logging and then sending a INVITE that includes a badly formatted CallId, which is the maximum allowed size for a CallId:
"set global callTrace errorFilter errorType parseError"

The code is modified to not overwrite the end of the buffer when it is attempting to write a very long error message string.

Workaround: Disable the parseError tracing.

A BYE message was sent to the wrong port.

Impact: The SBC does not update remote connection address when refresh INVITE is received from different port.

Root Cause: When the endpoint is registered over TLS initiating a call and after the call is established, the endpoint sends a refresh register from a modified port. When a refresh INVITE is sent with no SDP change, the SBC does not send BYE to a modified port when the call disconnects.

Steps to Replicate: 

1. Register endpoint over TLS.
2. Endpoint initiates a call, call gets connected.
3. Endpoint sends refresh register from modified port.
4. Endpoint sends refresh re-INVITE from modified port.
5. Called party disconnects the call.
6. Verify that the SBC sends BYE to modified port towards caller.

The code is modified to ensure the SBC updates the remote address when refresh INVITE is received from modified port. This ensures when the registration is required and for a TLS connection, after the call is released BYE is sent to correct port.

Workaround: N/A

The SMM for changing ISUP message inside SIP-I does not work properly.

Impact: When a message is received with mime content having single msgbody, then after the SMM application on the msg body, the SBC re-formats it as a normal msgbody instead on the mime content. The re-formating was not handled properly.

Root Cause: When a message is received with mime content having single msgbody, then after the SMM application on the msg body, the SBC re-formats it as normal msgbody instead on mime content. 

Steps to Replicate: 

1. Add the SMM rules that applies on the message body.
2. Send message with mime content having single msgbody in it.

The code is modified so that whenever the SBC re-formats the mime body after the SMM application it formats correctly.

Workaround: N/A

Portfix SBX-101294 (Originated in Release 7.2.5): Intermittently, the SBC SWe accesses the sipSigPort for sending the SIP messages to the core side.

Impact: The relay options with registration may route to the wrong Sip Signaling Port and causing call fails from AS to IAD.

Root Cause: When the IAD sends a registration from SSP1 and relay options from an other SSP2. Since the options from the SSP2 do not match with RCB from SSP1, the SBC treats the options from the AS, triggering an invalid routing to the wrong destination.

Steps to Replicate: Configure the SSP1 and SSP2, and enable maskRcbPort. The registration from SSP1 succeeds. Options from the SSP2 will trigger a relay back to the IAD (not AS). Subsequent calls from AS may fail or refresh registration.

The code is modified to detect the relay messages from the IAD or AS based on RCB looks up (not just based on SSP).

Workaround: Disable the maskRCBPort. Or use the SMM to make the SBC respond to OPTIONs (not relay).

Portfix SBX-100479/480: Observed DSP and MAJOR logs coredumps on the 7000 Platform while running the ICM Scaling suite.

Impact: The DSP and MAJOR logs coredumps on the 7000 Platform.

Root Cause: Certain variables used in the interrupt context were not protected while being modified outside the interrupt. This impacted the inter-core health-check logic at the DSP and incorrectly led to an error being reported, eventually leading to a coredump.

Steps to Replicate: From an HA Pair of fully loaded SBC-5100/5200/5110/5210/5400/7000

1. Run a transcode load.
2. Initiate a switchover.

The code is modified so variables are now interrupt-protected.

Workaround: N/A

Portfix SBX-101547 (Originated in Release 8.2.2): There was a switchover and coredump after call placed on hold.

Impact: The code was dereferencing a null pointer and causing the SCM process to crash.

Root Cause: The customer had the passThruPrivacyInfo control enabled on both the ingress and egress privacy profiles. But the ingress side of the call did not pass through some expected information related to the INVITE message and it caused the code to dereference a null pointer.

Steps to Replicate: Unable to reproduce this issue, the code has been fixed based on code review and coredump analysis.

The code is modified to validate the pointer is not null before trying to use it.

Workaround: Disable the passThruPrivacyInfo controls.

The call is failing intermediately due to the Unsolicited Call Cleanup.

Impact: The call is failing intermediately due to the Unsolicited Call Cleanup

Root Cause: The affected call failed due to the XRM having received an error response from the NP for modify RID command, which was most likely the race condition in the NP. This issue occurred because the RID enable and modify commands were issued within the same seconds but from two different NP users.

Steps to Replicate: Due to the nature of race condition, it is not guaranteed to reproduce the problem.

The code has been fixed based on a code review.

The code is modified to use the same NP interface user handle when sending the RID enable and modify commands to the NP for that special case.

Workaround: N/A

Portfix SBX-99579 (Originated in Release 9.0.0): Observed the Ema process core dump on the S-SBC during a sbxrestart. 

Impact: The EMA process may coredump and the application may not come up after an upgrade if the SWE/Cloud has a cinder volume attached for log storage.

Root Cause: While upgrading the SWE/Cloud platform attached with cinder volume, file ownership and permissions in the cinder volume are untouched causing wrong set of ownership on the EMA log file resulting in the EMA failing to start.

Steps to Replicate: Perform an upgrade from 7.x to 8.2.3R0 in cloud with cinder volume and check the sbxstatus and check for coredumps. With the problematic build, the EMA coredumps can be observed.

The code is modified so during an upgrade, the EMA log dir ownership is set to the sonusadmin:sonus to ensure the EMA process can access the log file and can start without issues.

Workaround: Manually change the ownership of the EMA log dir after an upgrade with below command:
chown -R sonusadmin:sonus /var/log/sonus/ema/

Portfix SBX-102006 (Originated in Release 7.2.5): The SBC failed to send ACK to the right IP.

Impact: After a call was connected with egress received RR in 2xx, the SBC sends a reInvite, and received the rexmit of 2xx. Later on, the SBC sends ACK to the wrong destination.

Root Cause: A rexmit of 200OK, accidentally deletes the previous RR.

Steps to Replicate:

1. A calls B.
2. B responds with a 200OK with RR.
3. A reInvite triggers the SBC reInvite to B.
4. B responds with 2xx and rexmit 2xx.
5. The SBC sends ACK to the wrong destination (based on contact not from previous RR received).

The code is modified so the SBC ignores updating the data of the dialog.

Workaround: N/A

Portfix SBX-99020 (Originated in Release 9.0.0): Observed Major logs with the error prints of 488 on the SBC 5210 platform while running direct media.

Impact: The calls stop working as LIF bandwidth is not freed.

Root Cause: In the case of a multi stream direct media call with the last media stream as non-RTP stream, the SBC does not release the session bandwidth even when the call is released. This leads to a leakage in interface bandwidth resulting in call failures after some time.

Steps to Replicate: Run direct media calls with multiple streams with the last media stream as a non-RTP stream (BFCP, MSRP).

The code is modified to release the session bandwidth even when the last media stream is non RTP stream in a direct media call.

Workaround: N/A

The call forking with the MS Teams and GW-GW fails.

Impact: The call forking with a GW-GW to might not work correctly.

Root Cause: The call forking logic was not copying all of the internal parameter information when grouping the routes based on each AoR tag. This meant the SRTP enabled information was lost and the call was initiated using plain RTP instead of the SRTP.

Steps to Replicate: Make a call forking call over GW-GW where one egress end point requires SRTP and the other does not. Check that the SRTP call is correctly setup.

The code is modified to correctly copy all of the route parameters between the PSX response and the forking queues.

Workaround: N/A

Portfix SBX-95873 (Originated in Release 7.2.2): The E-SBC JIAD622 crashed.

Impact: The problem occurred when the Version 3 T.38 fax is enabled. In this case, a CED 2100 Hz is sent to the SBC to G711 leg while it receives the T.38 ANSAM packets from T.38 side, the DSP hardware stack can crash.

Root Cause: The root cause has the incorrect initialization in 3rd party T.38 stack.

Steps to Replicate:

1. The SBC should be configured for the ingress as G711 and egress G711 with T.38 for Version 3.
2. Also configure ingress PSP as transcode only and make sure that codecs allowed for transcoding for ingress has G711 and egress has both G711 and T.38.
3. Make a SIPP call. Before adding the fix, the DSP cores and reloads.

The code is modified to avoid crash.

Workaround: N/A

The SBC Software Edition SWe was not sending media to the MS Teams client.

Impact: When there are more than 10 MS teams endpoints that are potential media candidates for an outgoing ICE call to MS teams, in some cases ice learning may not complete resulting in media path not being enabled when the call is answered.

Root Cause: For each outgoing call, the SBC has an internal nominated candidates list of up to 10 entries for storing ICE information received in stun requests from potential media candidates (which can be received before response with SDP) . On receiving SDP, stored entries are checked to see if ICE has been learned for the media stream in the SDP. However, if the SDP media is from the 11th or higher endpoint to have sent a stun (which would not have been stored) that can result in ICE learning not completing.

Steps to Replicate: 

1. Initiate a PSTN to MS teams ICE call through the SBC.
2. Before 200 OK with SDP from MS Teams, send ICE stun requests with use candidate from 80 different endpoints.
3. Answer the call from the 80th endpoint to have sent a stun (so 200 Ok from MS teams will have 80th endpoints media stream)
4. Verify that voice media flows successfully in both directions on the call.

The code is modified to allow for up to 80 endpoints as potential media candidates.

Workaround: None if more than 10 teams potential endpoints are necessary for a call, but works fine for up to 10.

The SBC 7000 has wrong value for call duration

Impact: The call service duration (CDR field numbers 10 and 14) has the wrong values causing billing issue.

Root Cause: This issue is seen when there is mid call message in this case session refresh. Due to this session refresh, there are mid call messages(INV/200OK/ACK) happens end to end. This makes the "callServiceEstTime" in the function CcRelayInfoMsg() overwritten every time causing the CDR fields 10 and 14 to have wrong values.

Steps to Replicate:

1. Enable IPSP flag "End to End ACK" on both TGs.
2. Disable the flag "No CDR Change in End to End Ack" .
3. Make a call from A to B and let the call be connected with session refresh enabled.
4. Once the session refresh happens, disconnect the call and check the CDR.

The code is modified in the function CcRelayInfoMsg() so that the "callServiceEstTime" does not get overwritten once it is recorded initially during the initial offer-answer.

Workaround: 1. Enable the flag "No CDR Change in End to End Ack".

The DBG file was filling up with messages "SIPCM: *ThreadPool: messageSequence".

Impact: The DBG logs can be overrun with "SIPCM: *ThreadPool: messageSequence" messages.

Root Cause: The DBG logs can be overrun with "SIPCM: *ThreadPool: messageSequence" messages, when there is a double CRLF "pings" received by the SBC over UDP transport.

Steps to Replicate: Send Double CRLF "pings" over UDP to the SBC.

The code is modified to properly dispose of Double CRLF "pings" received by the SBC over UDP.

Workaround: Inhibit the transmission (or reception) for Double CRLF "pings" over UDP.

There was a malformed OTG after an upgrade to 7.2.4R2 on GW-GW calls.

Impact: In a SIP-GW-GW-SIP call, if the ingress SIP INVITE contains "tgrp" in the Contact header, the contents of the "otg" parameter in the egress SIP INVITE will be incorrect.

Root Cause: Design was using an incorrect pointer value when creating the OTG parameter.

Steps to Replicate: In the "Egress IP Signaling Profile", set the "Include OTG" in Originating TrunkGroup Options field.

1. Send a SIP-GW-GW-SIP call with a Contact Header that includes a "tgrp" parameter.
2. Look at the contents of the "otg" parameter in the "From" header in the egress INVITE. It should be garbage.
3. Load up code with fix.
4. Run the same call (with the same "Include OTG" configuration.

The "otg" in egress INVITE should be the originating TG on the originating GW.

The code is modified to use the correct pointer when creating the OTG parameter.

Workaround: Do not send the "tgrp" parameter in the ingress INVITE.

Portfix SBX-102029 (Originated in Release 8.1.0) There was an issue with Packet loss calculation on the NP.

Impact: The packet loss count for for a call is incorrect when packets are lost at the end (last 32 packets) of the call.

Root Cause: The packet loss algorithm did not include checking the last 32 packets of a call.

Steps to Replicate: Send an RTP stream that includes skips in the sequence number during the last 32 packets.

The code is modified to check if packets are lost in the last 32 packets.

Workaround: N/A

The P-Early-Media header with sendrecv sent back to the ingress before the SDP established.

Impact: When the makeInBandToneAvailable is disabled, on a Tone And Announcement profile, the SBC sends 180 Ringing with PEM header as sendRecv even when egress 180 has no SDP. This causes ring back issues on ingress endpoints.

Root Cause: When the makeInBandToneAvailable is disabled, the SBC does not include SDP in 180 message sent to ingress but still sends PEM header as sendRecv.

Steps to Replicate: With a fix, the SBC is sending PEM header as inactive when makeInBandToneAvailable is disabled and egress 180 Ringing has no SDP.

The code is modified to ensure the SBC sends a PEM header as inactive when the makeInBandToneAvailable is disabled and egress 180 Ringing has no SDP.

Workaround: Enable makeInBandToneAvailable on Tone And Announcement Profile.

The SBC CLI responds significantly slower while entering the ping command when one or more options are specified.

Impact: Attempting to run the SBC CLI commands ping, ping6, traceroute, traceroute6 or aaarule-display-generatecli with multiple parameters causes the CLI session to freeze.

Root Cause: Parsing of parameters on these CLI commands not correct.

Steps to Replicate: Try to run the following command from CLI:
ping -I mgt0 -c 5 1.2.3.4

The code is modified to address the issue.

Workaround: Use Linux equivalents to the commands, rather than using the SBC CLI or, if using the SBC CLI, limit the number of parameters.

The SIP-I calls cause a platform switchover when the Map Calling party to the PAI is set.

Impact: The SBC may core when the configuration sipSigSrvcGrpURIPreference SIP on the egress but the incoming has tel and has pai (tel).

Root Cause: The SBC accesses the invalid diversion address.

Steps to Replicate: This is specific test case from customer, and cannot be reproduced.

The code is modified to validate the diversion address before access to it.

Workaround: Disable the feature sipSigSrvcGrpURIPreference.

There was an SAM process core dump with v06.02.01 F012.

Impact: The SAM and PRS process cored in the Standby node.

Root Cause: The DRBD split brain lead to DRBD GI reset. This leads to the DRBD full sync, increasing the i/o congestion and ultimately causing a healthcheck timeout.

Steps to Replicate: To test, run "drbdadm disconnect mirror". Once getting a "standby split-brain" log in DBG logs, run "drbdadm disconnect mirror" again so that DRBD does not connect automatically and getting into the block that needs to tested. Run "drbd-overview" and check if drbd does not went for full sync.

The code is modified to perform DRBD checks after a DRBD split-brain recovery.

Workaround: N/A

There was a double SBC SCM process coredump.

Impact: The SBC may core for an antiTromboneData feature.

Root Cause: The SBC may access null pointer in antiTromboneData feature.

Steps to Replicate: A call B, B(sdp) loop back to the SBC (Bprime). antiTromboneData is enable for B and Bprime.

NOTE The Bprime also has a replaces header to replace a call which does not exist in the SBC, causing SBC reply 481.

The code is modified to validate pointer before access and address the issue.

Workaround: N/A

The SBC SWe 8.2.0F1 duplicates audio from call B to call A.

Impact: If CN is not negotiated for G711 codec and remote peer still sends CN packets that match default CN payload type (13), then user on other end may hear cross talk audio of completely unrelated channel or hear own reflect audio.

Root Cause: When a g711 side does not negotiate CN in signaling, DSP does not initialize Comfort Noise Generation object. However, if remote peer still sends CN packet that matches the g711 SID payload type configured in PSP (default 13), DSP accepts the packet. It processes that CN packet incorrectly and as a result uses stale voice buffer which happens to be of another channel. This continues until next voice packet for that channel arrives. Hence, we happen to see cross talk audio from other call during silence period. In some cases, the user may hear own audio also and that is different manifestation of the same problem. This issue is specific to SWe.

Steps to Replicate: 

1. Run the CN in signaling.
2. The DSP will not initialize Comfort Noise Generation object.
3. Send a remote peer to the CN packet that matches the g711 SID payload type configured in PSP (default 13) and the DSP accepts the packet.
4. Process that CN packet and as a result, uses stale voice buffer that happens to be of another channel.
5. This continues until next voice packet for that channel arrives.

The code is modified to initialize the comfort noise object even though CN is not negotiated and process the CN packets correctly.

Workaround: There are multiple workarounds for this issue:

1. Change the default payload type of comfort noise from 13 to something else (say 15) in PSP of peer that is sending CN packets. This will make DSP drop CN packets because payload type will not match. Run the PSP configurationshow

configuration profiles media packetServiceProfile G711_NONE_NONE_NONE..silenceInsertionDescriptor {g711SidRtpPayloadType 13;heartbeat enable;}

2. Enable the silence suppression on PSP of peer that is sending CN packets and keep CN payload type that matches with CN payload type.

A memory leak was observed in Sam process.

Impact: There was a memory leak was observed in the SAM process.

Root Cause: There is race condition in the code that handles CALL_AUDITs and CALL_CLEANUPs that can cause a memory leak.

If the response to a CALL_AUDIT takes to long, the code that handles the "late" response has allocates memory for a structure that may never get freed.

Steps to Replicate: This issue is caused by a race condition that cannot be forced and therefore we cannot specify steps to reproduce this issue.

The code is modified to add a timer every time a fault structure is allocated. When the timer expires, if the the structure still exists it is freed.

Workaround: N/A

The SBC was reporting REGISTER_PARSE_ERROR and replies with SIP 500 when it receives retransmitted initial REGISTER (CSEQ 1) followed by a REGISTER with Authorization header (CSEQ 1).

Impact: When the SBC recv rexmit of registration (cseq=1) after sending 401/407, the SBC relays to the AS. At the same time, the IAD sends new registration(cseq=2) for authentication. It triggers race condition to the SBC and responds with 500.

Root Cause: The SBC should relay the rexmit (cseq=1) registration to the application server.

Steps to Replicate: IAD send registration(cseq1) to AS, AS response 401 to IAD, IAD rexmit the same cseq1, and send a new cseq2 with auth header.

The code is modified to respond to the rexmit request (cseq=1) so that a subsequent one cseq=2 can handle properly relay to the AS.

Workaround: N/A

The SIPFE lookup returned the rcb pointing to wrong SCM post-upgrade.

Impact: After an upgrade, registrations mirroring on SIPFE is corrupted. As a result, incoming traffic for the AOR may route to the wrong SCM and get rejected by the SBC.

Root Cause: The packing redundancy data structure that was sent to the standby were incorrect due to word alignment issue (using sizeof() to pack the data).

Steps to Replicate: Multiple of the same AOR registers from a different source. After an upgrade, call for register end point may route to the wrong SCM.

The code is modified to re-implement the packing logic and avoid using sizeof() to calculate the size of data structure.

Workaround: N/A

Portfix SBX-96001 (Originated in Release 9.1.0) The SBC 7000 has the wrong value for call duration.

Impact: The call service duration (CDR field numbers 10 and 14) has the wrong values, causing a billing issue.

Root Cause: This issue is seen when there is a mid call message in this case session refresh. Due to this session refresh, there are mid call messages(INV/200OK/ACK) that happen end to end. This makes the "callServiceEstTime" in the function CcRelayInfoMsg() overwritten every time, causing the CDR fields 10 and 14 wrong values.

Steps to Replicate: 

1. Enable IPSP flag "End to End ACK" on both TGs.
2. Disable the flag "No CDR Change in End to End Ack".
3. Make a call from A to B and let the call be connected with session refresh enabled.
4. Once the session refresh occurs, disconnect the call and check the CDR.

The code is modified so that the "callServiceEstTime" does not get overwritten once it is recorded initially during initial offer-answer.

Workaround: Enable the flag "No CDR Change in End to End Ack".

Portfix SBX-103948 (Originated in Release 8.2.1) The SBC Application is crashing when the CPaaS to PSTN call is made.

Impact: The SBC cores after a reboot/upgrade.

Root Cause: The issue was introduced when the SBC tries to support the 10000 SMM rules in 8.2.0.

Steps to Replicate: Configure the customer rules (delete sdp line). Major logs trigger but the rule still runs successfully. After a reboot/upgrade, the SBC cores.

The code is modified to fix the issue.

Workaround: Disable the delete sdp rule.

The EMA GUI is not showing the username of configured AD Domain Controller correctly.

Impact: Getting a Defined type NULL if it is "Direct type".

Root Cause: The EMA GUI is not showing the username of configured AD Domain Controller correctly.

Steps to Replicate: 

1. Log on to EMA
2. Navigate to Global > Servers > Domain Controller > Domain Controller ID
3. See the All the user Names in the screen.
4. Now the interface will the exact names that was assigned.

Platform/Feature: SBC

The code is modified to get the correct Defined Type.

Workaround: N/A

SDP attribute a=X-nat - duplicated (selective sdp relay).

Impact: Due to a new change in 8.2 we did not consider the b line length properly, so "a=X-nat" was added twice

Root Cause: Due to a new change in 8.2, we did not took the b line length into account while adding b line on egress. So we ended up adding "a=X-nat" line twice, once as part of b line and once as a line

Steps to Replicate: 

Send INVITE with b line and a line in SDP as below

v=0
o=Sonus_UAC 203537 229017 IN IP4 10.xxx.xxx.xxx
s=SIP Media Capabilities
c=IN IP4 10.xxx.xxx.xxx
b=AS:84
t=0 0
a=X-nat:0
m=audio 1086 RTP/AVP 0
a=rtpmap:0 PCMU/8000
a=sendrecv

"a=X-nat " line should not be added twice on egress side.

The code is modified to copy exactly b line bytes as part of adding b line on egress. So "a=X-nat" line is added only once as expected

Workaround: N/A

Portfix SBX-100839: The GR-HA leader election could choose the starting node that is not in sync.

Impact: A race condition exists with the G-HA leader election algorithm whereby when coming out of split-brain so you could choose the node that is starting and not sync'd to be the leader. This causes a full outage.

Root Cause: The potential existed to check the wrong node's sync state when verifying the potential leader was in sync.

Steps to Replicate: Cluster is configured for enhanced leader election.
Both nodes are up and then a switchover so that the standby is promoted to active. While the active is coming up, force a split brain and then re-establish communication between the nodes. Verify that the booting node is chosen to restart even though it is the node that was active the shortest amount of time.

The code  is modified so that the proper nodes sync state is checked.

Workaround: N/A

Portfix SBX-100481: Observed a jitter for more than 5ms in the passthrough call load.

Impact: More than a 5ms jitter and relatively high packet loss was observed in passthrough calls.

Root Cause: There was a segregation of media and non-media processes at initialization time that may fail occasionally, leading to non-media processes landing on vcpus meant for media processing.

This leads to a higher jitter and possibly higher packet loss.

Steps to Replicate: Issuing the following command show only yield kernel threads and SWe_NP processes:
cset proc -l root

The code is modified to function reliably.

Workaround: Reboot the instance.

Portfix SBX-99414: The 6ms of jitter is observed on the pktart side from the SBC while the accepted value of the jitter is 3ms for a transcoded call in a call mix load of direct media and a transcoding scenario.

Impact: More than 6ms of jitter was introduced in transcoded calls in call mix scenarios.

Root Cause: The primary and Secondary ATA Interrupts landing on media processing cores were causing processing jitter due to high volume of preemption.

Steps to Replicate: Verify that the IRQ 15 and 14 are incremented against core 0 only.

The code is modified to avoid the media processing cores.

Workaround: N/A

The user creation/login validation issue in the EMA and Platform Manager.

Impact: When the username contain special characters for example: " - " , it will have problem to log into platform manager, but not in EMA. It also doesn't throw any error while creating user contain special character " - ".

Root Cause: EMA username supports underscore(-), when creating a new user, but when logging in to a platform manager, the platform manager does not support underscore(-) and login throws error authentication mismatch.

Steps to Replicate: 

1. Login into EMA and create new user with username ribbon-eci.
2. Login into EMA and platform manager.
3. Platform manager login throws error.

The code is modified to fix the issue on the EMS supports.

Workaround: No workaround.

Portfix SBX-96565: The sbxstop on Active can lead to the /dev/drbd0 not mounted on new Active..

Impact: The sbxstop of Active can lead to /dev/drbd0 not mounted on new Active.

Root Cause: The Drbd was getting unmounted right after the mount when the switchover happens.

Steps to Replicate: The following are the steps that need to be followed to test this:

- On node A, run the switchover from platform manager
- B will become active, wait for A to become standby.
- Run sbxrestart on B so that A becomes active again.
- Check df or /proc/mounts for drbd mount status.

The code is modified so the cron job runs once whenever the node becomes active and mount the drbd partition.

Workaround: N/A

Potential memory leak of ICM message in SG FSM when switching to secondary NIF for MS teams call.

Impact: It is possible to get a small memory leak in an MS teams media optimization configuration for each PSTN to Teams call that uses the secondary (public) media interface to teams endpoint. Over a large number of such calls (estimate 10,000 +) this could result in enough memory loss to cause new calls to fail to establish.

Root Cause: When an ICE call to MS teams switches to the secondary media interface, the existing egress ICE context for the primary interface is replaced by a new one for the secondary interface. This context switching involves inter task messaging on the SBC, there was a potential path in the code where one of these messages was not releasing its used memory correctly.

Steps to Replicate: Establish and release a large number (10,000 +) of PSTN to MS teams calls that use the secondary (public) media interface to teams endpoints.

Platform/Feature: SBC

The code is modified to remove the inter task message that could have lead to the issue.

Workaround: N/A

The Standby SBC memory at 94% seems the SCM Process was leaking.

Impact: The SCM process on standby is running out of memory when the path headers are included in the Registration messages.

Root Cause: The SCM process on standby is leaking a SIP structure related to the path headers that are included in the Registration messages.

Steps to Replicate: This issue can be reproduced by running Registration load which includes path headers in egress Registration messages.

The code is modified to free the structure that had been leaking.

Workaround: N/A

The TEAMS zone detection should be case insensitive.

Impact: The MS Teams zone detection logic may not work to provide native support for functionality.

Root Cause: In the 8.2 release, the SBC was updated to provide native support for MS Teams functionality previously implemented using SMM. This is done by reading the FQDN from the pathcheck logic in the zone. It was assumed that the FQDN would be in lower case but some customers have configured in upper case and the code is not matching this format, which results in the native functionality not being provided.

Steps to Replicate: Configure all the FQDNs in the patchcheck configuration in upper case e.g. SIP.PSTNHUB.MICROSOFT.COM and the native Teams functionality will not be triggered.

The code is modified to perform the non case sensitive checking for the pstnhub.microsoft.com FQDN.

Workaround: Change the sip.pstnhub.microsoft.com FQDN in the pathchk configuration to use lower case instead of uppercase or leave SMM in place from older releases.

The AddressSanitizer has heap-buffer-overflow on the address 0x607000326fe4 at pc 0x563429a99aac bp 0x7f371d414210 sp 0x7f371d4139c0.

Impact: When the END2END ACK control is enabled and making calls between the SBC and GSX, the GSX sends a bad parameter to the SBC and it causes the SBC to read invalid memory and occasionally coredumped.

Root Cause: The problem is due to the GSX using the wrong enumeration range for an internal CPC parameter type. The associated parameter it creates does not contain any data. This causes the SBC to think its a completely different parameter which is expects to contain mandatory parameter data. It reads the data, which results in it reading off the end of the message block. This has been broken since original implementation and generally does not cause issues. But if the message memory block happens to be at the edge of the valid memory range then it can result in an invalid memory read which triggers a coredump.

Steps to Replicate: This issue is highlighted when using ASAN enabled images in the engineering lab, but it has been known to cause occasional coredumps in production.

The code is modified to ignore the bad parameter information coming from the GSX to avoid reading invalid memory.

Workaround: Disable the END to END ACK control if possible.

Portfix SBX-100211: The MIM Hash memory leak for an IMSLI call was in the DSBC only.

Impact: One of the modules(MIM) used for IMS Lawful Interception leaks a hash entry every time a IMS Lawful interception is triggered for a call.

Without this fix, there will be a continuous memory leak of an hash entry for each IMS Lawful intercepted call.

Root Cause: An IMS Lawful interception stop indication is not being propagated to One of the modules used for IMS Lawful interception as of a result of this, the hash entry created to achieve/keep track of IMS interception is not freed even after interception for that call stops.

Steps to Replicate: Make an IMS Lawful intercepted call.

The code is modified so that it can cleanup the concerned hash entries.

Workaround: N/A

The SYS is filling up the "mcsEncodeCPC_MSG_INFO_STR: CPC_OP_STR parameter length mismatch".

Impact: The following log message is filling up the SYS log when the STIR/SHAKEN feature is in use:

MAJOR .GWCM: mcsEncodeCPC_MSG_INFO_STR: CPC_OP_STR parameter length mismatch: sizeof length 152: parameter length 216, parameter:397

Root Cause: There is code in the SAM that is checking for an internal inconsistency and logging this message when it detects an internal inconsistency.

In this specific case, the message is being logged unnecessarily and it does not indicate any impact on customer functionality.

Steps to Replicate: Setup the SBC and PSX for STIR/SHAKEN call flows and run a GW-GW call.

The code is modified to ensure that this message is no longer logged for in this very specific scenario.

Note: This specific message (for parameter:397) does not indicate any impact on customer functionality.

Workaround: There is no workaround. 

During an SBC switchover, the SCM process coredumped as a result.

Impact: The SCM process cored due to double freeing of a SIP structure related to Subscriptions.

Root Cause: The SCM process cored due to double freeing of a SIP structure related to Subscriptions.

Steps to Replicate: The issue is not reproducible. 

The code is modified to ensure that it does not attempt to free the structure that has already been freed.

Workaround: N/A

Portfix SBX-99204 (Originated in Release 9.1.0) There was a TIPC log format change and needs to update CHM code for duplicate error detection.

Impact: Duplicate The TIPC address logic may fail to detect a duplicate TIPC address and nodes will fail to take the proper role and/or start.

Root Cause: The kernel message was being looked for has changed.

Steps to Replicate: Install both nodes as the primary node. When the second node is started, it should fail to start and report a standalone/HA pair configuration mismatch.

The code is modified to look for the proper error message.

Workaround: Proper install of the nodes as primary and secondary, and proper setting of the TIPC ID in a swe environment.

Portfix SBX-97644: The SBC is not sending the precondition parameter in Supported header in UPDATE message in 1-1-2 customer call flow in SBX-97644.

Impact: The SBC was not sending the preconditions option tag(in Require/Supported) in the UPDATE request generated towards the ingress endpoint during preconditions transparent settings.

Root Cause: While performing precondition transparency, the SBC uses the received message buffer to copy preconditions option tag transparently. But, since the UPDATE here was a locally generated one, the SBC was unable to get any option tag. Therefore precondition option tag was not sent.

Steps to Replicate: 

1. The Egress endpoint has to send the 183 (Dialog-2) with “Require: precondition” header to SBC.
2. The SBC has to send the 183 to ingress endpoint.
3. After PRACK and 200 OK, the SBC has to send the UPDATE message to ingress endpoint with “Supported: precondition” header. But the SBC is not sending the precondition parameter in supported header.

The code is modified for the outgoing message(to maintain as much as transparency). When sending such UPDATES, refer to this state and add option tag accordingly.

Workaround: One workaround is to add SMM.

Portfix SBX-97644: The SBC is queuing the UPDATE message during the preconditions and downstream forking call flow(customer 1-1-2 call flow) in SBX-97644.

Impact: The SBC is queuing the 200 OK of UPDATE with downstreamForkingSupport enabled and UPDATE received on the egress.

Root Cause: In case of Downstream forking queuing mechanism, there could be chance that TYPE_STATUS can fall-thrugh to CASE_OTHER so that when calling the generic CMD, the callDirection,msgtype and msgStatus are not being sent out.

Steps to Replicate: This behavior is not always reproducible. Sometimes, the SBC is able to process the UPDATE message and forward to egress leg and call is successful.

The code is modified to send callDirection,msgtype and msgStatus in this case

Workaround: N/A

The NVIDIA V100 Based GPU instance fails to come up in the VMWARE.

Impact: The NVIDIA V100 GPU Accelerated instance does not come up on the VMware host.

Root Cause: Due to large PCI memory requirements for V100 card, the V100 based VMware instance does not come up in legacy BIOS mode.

Steps to Replicate: Bring up the SBC VM with EFI BIOS instead of the legacy BIOS.

The code is modified so the V100 based VMware instance needs to be booted in EFI mode to work correctly.

Workaround: There is no workaround for this issue. V100 based VMware instance needs to be booted in EFI mode.

Portfix SBX-100561 (Originated in 9.1.0): "Fac Total Block Count Stats" page was not loading in the EMA.

Impact: "Fac Total Block Count Stats" page was not loading in the EMA

Root Cause: In the SIPFE application, the "Global_facTotalBlockCountStats_keybuf" object was not registered with ICM. This led the ICM message send to SIPFE fail due to this EMA query failed.

Steps to Replicate: 

1. Launch EMA.
2. Go to All -> Global -> Fac Total Block Count Stats.

The missed object registration is done.

Workaround: N/A

Portfix SBX-100642 (Originated in Release 9.1.0): The cloud instantiation is failing on the M-SBC2 as the LCA exits with an error.

Impact: In the SBC Cloud environment, the LCA fails to come up due to index exchange errors.

Root Cause: There was a race condition between LCA read and index-serf write on the platform.json.

Steps to Replicate: This issue is rarely observed.

* To reproduce the issue, launch a 4:1 I-SBC instance on Openstack, in such a way that a RG serf detects a service ID collision.

* This results in reboot of the instances.

* In the next boot, the LCA started the index exchange serf.

* Index exchange serf will be started and comes up in the running state.

* Index exchange serf detects the presence of a peer in the cluster, the index exchange serf tries to update the platform.json file.

* At the same time, the LCA module(setCPUIndices.py) tries to read the content of the platform.json file.

* The json load operation failed, since the content is not in proper json format.

* LCA exits out. The serf operations continues. Application will not be brought up.

The code is modified so the lock mechanism is reading and writing process of platform.json file.

Workaround: Reboot of the SBC instances which encounters this issue.

Portfix SBX-86763 (Originated in Release 9.0.0): The sipClientStatistics gets incremented on the SLB by 2 for a single call.

Impact: The sipClientStatistics gets incremented by 2 for a single call.

Root Cause: This issue is seen only after a switchover of SLB. After a switchover, there were two calls to register to the sipClientStatistics object because for each call, the statistics would get updated twice from the two registrations.

Steps to Replicate: Make a call before and after switchover of SLB and ensure that sipClientStatistics is updated properly

The code is modified to register only once to the sipClientStatistics object.

Workaround: N/A

Portfix SBX-99197 (Originated in Release 9.0.0): The ASAN setup CE_Node2.log file has errors.

Impact: There was invalid memory access while allocating the media tap resource for an audio stream.

Root Cause: There was an invalid index is used to access the array which caused this out-of-bound memory access.

Steps to Replicate: 

1. Provision the SIPRec for a basic audio call.
2. After the egress peer answers, the SBC sends INVITE towards a SIPRec server.
3. SIPRec server answers the call.
4. At this stage, the SBC allocates media tap related resources and that caused this invalid memory access.

Access the array only if the index is valid to fix the issue.

Workaround: N/A

Portfix SBX-98418 (Originated in Release 9.0.0): Deleting the system->slb->commInterface was not working as expected.

Impact: Deleting the /system/slb/commInterface was not working. Connections between the SBC and SLB still remain active.

Root Cause: The delete operation on the tree /system/slb is not handled separately as of now. For all operations on /system/slb, the cdb_get was being done and would fail for during the delete operation. So deletion was not being handled properly.

Steps to Replicate: Delete the system->slb->commInterface. Ensure the connections between the SBC and SLB are closed.

The code is modified to handle deletion of system->slb→commInterface.

Workaround: Disable and enable the SLB usage if the commInterface is changed. Disabling of SLB usage requires an SBC restart.

Portfix SBX-97102 (Originated in Release 9.1.0): During a direct dial to call queue and then while transferring to another agent, there was no RBT heard on the PSTN side.

Impact: While making a call from PSTN user to Teams Call Q when the Teams1 user later transfers the call to Teams2 user, there is no RBT heard.

Root Cause: Call control was not in the correct state to generate the RBT on the subsequent transfer from Teams1 to Teams2.

Steps to Replicate: Make a call from PSTN user to Teams Call Q, answer with Teams1 user and then transfer to Teams2 user and check that RBT is heard during the transfer.

The code is modified to ensure that call control transitions to the correct state in order to generate the RBT on the latest transfer.

Workaround: N/A

Portfix SBX-101156 (Originated in Release 9.1.0): The SRTP context for video is omitted.

Impact: The SBC offers RTP context for video stream instead of SRTP during RESUME re-Invite after HOLD is performed with video mediaPort being zero.

Root Cause: Certain code was copying the SRTP info from previous active SDP in order to retain the same SRTP key in subsequent call modifications.

However, it does not handle the case if that particular stream is removed in between and then was added back.

Steps to Replicate: 

1. Setup Audio and Video RTP to SRTP pass-Thru call.
2. Ingress peer (RTP side) sends re-Invite to disable video stream (sends re-Invite with audio stream having valid media port and sendrecv AND video stream having port=0 and inactive).
3. Ingress peer (RTP side) sends re-Invite to add video stream back (sends re-Invite with audio stream having valid media port and sendrecv AND video stream having valid media port and sendrecv).

Expected Result:
a) RTP-SRTP A+V call gets established.
b) The SBC disables video stream and sets up RTP-SRTP call with audio stream.
c) The SBC re-establishes Audio and Video RTP-SRTP call.

Actual Result:
a) RTP-SRTP A+V call gets established.
b) The SBC disables video stream and sets up RTP-SRTP call with audio stream.
c) The SBC re-establishes Audio and Video RTP-SRTP call.

The code is modified to copy SRTP info from previous active SDP only if stream is valid in that SDP.

Workaround: N/A

Portfix SBX-99169 (Originated in 9.0.0): The ASAN heap-buffer-overflow in the SipFeProcessSlbIncomingPdu - SBC ASAN build failure when testing epcac DBL with SLB.

Impact: The SBC was accessing call control memory block after the memory block had been freed.

Root Cause: The call control logic maintained a queue of call control blocks that had outstanding events, which needed processing. However, in some places the code processed the outstanding events and did not remove the call control block from the queue. While processing call cleanup events, it was possible that the same call control block got added to the queue twice. When the call control instance was released it removed one instance from the queue but later processing code then noticed there was still a call control block left in the queue and tried to read the memory to process it after it had been freed.

Steps to Replicate: This issue is only highlighted in engineering lab while running with ASAN enabled images. Run call load and trigger bulk call failure.

The code is modified to ensure that call control blocks are removed from the pending queue, when all outstanding events are processed to avoid accessing memory after it is freed.

Workaround: N/A

Portfix SBX-101355 (Originated in Release 9.1.0): The SBC was not stable after the upgrade to 8.2.2R0 - serf.conf file corruption RCA.

Impact: On an upgrade of the SBC SWe to version 8.2, the primary upgrade status was not updated causing post-upgrade checks failure.

Root Cause: This issue was due to the missing peerHAIp entry(100.xxx.xx.x) in serf config file on primary node.

Steps to Replicate: Perform upgrade to the SBC SWe to fix version 8.2.2R1 and ensure HA IPs are updated in serf.conf file.

The code is modified to ensure the HA IPs of both nodes are added to the serf conf file.

Workaround: N/A

Portfix SBX-100454 (Originated in Release 9.1.0): The SBC fails to transcode between the G711X in certain SIP - GW2GW scenarios with HDCodecPreferred and preferNBPassthruOverHDTranscode flags enabled.

Impact: The SBC fails to transcode between the G711X in certain SIP - GW2GW scenarios with the HDCodecPreferred and preferNBPassthruOverHDTranscode flags enabled.

Root Cause: If the HD flag is enabled, the ingress GW is using the egress codec as selectedCodecEntry instead of codec entry from audioEncode array. However, the selected Codec Entry is not fully qualified since it's not intersected with Route PSP.

Steps to Replicate: SIPA - SBC1 (Gw-Gw) SBC2 - SIPB

1. SIP A is offering G711U along with other codecs
2. SIP B is answering with G711A. Ingress Route PSP -> G711U,G729A,G722
   Gateway Route PSP -> G711U,G729A,G711A,G722
   Egress Route PSP -> G711U,G711A
3. When SBC01 receives answer with G711A codec, it fails to transcode the call between G711U on the ingress SIP call leg and G711A on the egress GW2GW call leg. call is disconnecting with 488 error.

The code is modified to use the matching codec entry from audioEncode array instead of the selected codec in answer PSP on ingress GW.

Workaround: Testcases:

Case-1. Disable diff in SS flag and keep HD flags enabled.

Case-2. Disable HD flags and keep Diff in SS flag enabled.

Enabled both HD and diff in SS flag:

Case-3: Ingress and Egress Route PSP has G711USS,G711ASS respectively and GW route PSP has G711-DEFAULT.

Case-4: Ingress and Egress Route PSP has G711USS,G711ASS respectively and GW route PSP has G711SS-DEFAULT.

Portfix SBX-101894 (Originated in Release 9.1.0): The GCP: RTCP packets are not being sent and received by the SBC in UDP SRTP - UDP SRTP call scenario.

Impact: With a small SWe SBC profile in the GCP platform, when the SecureRTCP is enabled, RTCP packets were not generated/relayed by the SBC.

Root Cause: With the dynamic range of resources allocation based on SWe capacity model, media enc context range check was using a wrong value in packet processing, resulted in RTCP packet drops which required encryption with in the SBC.

Steps to Replicate: With GCP platform small SWe configuration, enable the RTCP relay/termination with SRTCP support and check the RTCP packets flow.

The code is modified to the right media enc range value for the SBC media packet processing to encrypt the RTP, RTCP packets.

Workaround: Enable RTCP relay/termination without SRTCP.

There was a race condition in handling of ccbPtr→bIsTonePlayingEgressFlag.

Impact: The SBC was incorrectly mapping re-INVITE with a=inactive to a=sendonly when 200OK arrives quickly after 180 without SDP and RBT configured.

Root Cause: When the ingress side is playing a tone it sends a message back to the egress side to let it know. The egress side is meant to clear this flag when the 200 OK message arrives. However, there was a race condition where the 200 OK could arrive before the ingress side is able to send the tone indication message to the egress side and then the flag is not reset.

The egress side uses this flag to update the SDP media direction to a=sendonly, when a hold indication arrives and tone is being played to allow the SBC to finish playing out the tone.

Steps to Replicate: With the SBC configured to generate ring back tone on the receipt of 180 without SDP. Make a call where the SBC sends out INVITE and the egress side responds with 180 without SDP and immediately followed by a 200 OK. Once the call is answered then send in an INVITE with a=inactive from the egress side and check that the SBC sends out INVITE with a=inactive on the ingress side.

The code is modified to no longer set the tone playing flag at the egress side if the 200 OK message is received.

Workaround: N/A

Portfix SBX-99561 (Originated in Release 9.1.0): The ASAN heap-use-after-free in CcProcCallFsmMsg - CLONE - The SBC ASAN build failure when testing the epcac DBL with SLB.

Impact: The SBC was accessing call control memory block after the memory block had been freed.

Root Cause: The call control logic maintained a queue of call control blocks which had outstanding events which needed processing. However, in some places the code processed the outstanding events and did not remove the call control block from the queue. While processing call cleanup events, it was possible that the same call control block got added to the queue twice. When the call control instance was released it removed one instance from the queue but later processing code then noticed there was still a call control block left in the queue and tried to read the memory to process it after it had been freed.

Steps to Replicate: This issue is only highlighted in engineering lab while running with ASAN enabled images. Run call load and trigger bulk call failure.

The code is modified to ensure that call control blocks are removed from the pending queue and when all outstanding events are processed to avoid accessing memory after it is free.

Workaround: N/A

Portfix SBX-96518 (Originated in Release 9.0.0): There was a memory leak that was detected for x headers.

Impact: Customers using the X-header capability on the SBC would see a small memory leak every time the SBC is creating the reason header based on the release information.

Root Cause: The could was double allocating a memory block by mistake resulting in a memory leak.

Steps to Replicate: Run a test case with the Japan X-header feature enabled and release the call so the SBC has to generate a reason header based on the internal cause value information to send along with the other X-headers.

The code is modified to avoid the memory leak.

Workaround: N/A

Portfix SBX-99115 (Originated in Release 9.0.0): Observing the MAJOR log "XrmMflowCmdSnoopBld cannot find Snoop Id 0" while running the SIPREC with 4 recorders on KVM-20vcpu setup.

Impact: Observing MAJOR log "XrmMflowCmdSnoopBld: Can't find Snoop Id 0" while running SIPREC with 4 recorders on KVM-20vcpu setup.

Root Cause: During the Modify, there is no check if the snoop is enabled before invoking resulting in ERROR log mentioned in the Jira.

During the Activate, there is a check for snoopId being enabled, missing during the Modify.

Steps to Replicate: Issue was observed with load testing with the SIPREC enabled.

Check if snoop is enabled for the modify flows to address the issue.

Workaround: N/A

The AddressSanitizer detected a heap-use-after-free on the address 0x6110000a302c at pc 0x55bcb2c39996 bp 0x7fbf04828250 sp 0x7fbf04828248 READ of size 2 at 0x6110000a302c thread T9.

Impact: The "heap use after free" occurs when IP Peer is created.

Root Cause: This is because of accessing the memory which is already freed.

Steps to Replicate: Re-Create an IP Peer with the same name, Ip Address and IP Port to reproduce this issue.

The code is modified to fix this issue.

Workaround: Do not create same IP Peer with same Name, Ip Address and IP Port.

If required, delete the old Ip Peer and re-create the same.

Portfix SBX-101830 (Originated in Release 9.1.0): The SBC services are going down while running the CLI to create toneCodecEntry.

Impact: The stack buffer overflow while executed on the toneCodecEntry for the AMR files.

Root Cause: This issue is from day one since USHORT was used instead of the ULONG for coding rate variable.

Steps to Replicate: Execute the toneCodecEntry CLI for AMR codec.

The code is modified to ulCodingRate(ULONG).

Workaround: N/A

Portfix SBX-99996 (Originated in Release 9.0.0): Observing the Major Logs in DBG logs while running the SIPREC on SBC 7000 and SBC 5400 setups.

Impact: Observed the following code mentioned MAJOR logs when running the SIPREC load on HA setup.

117 05082020 152048.782437:1.01.00.09099.MAJOR .XRM: *NpMediaPnpsNpCmdSend: Cmd 0x29 failed, error code 0xffffffff
100 05082020 152048.782532:1.01.00.09100.MAJOR .XRM: *NpMediaFlowModify: Failed status 0xffffffff

Root Cause: To handle scenarios where mirrored snoop context is transitioning from disabled to enabled.

Modify the snoop even though snoopId is DISABLED.

By directly setting NP_MEDIA_FLOW_MOD_RTP_SNOOP_FLAG | NP_MEDIA_FLOW_MOD_RTCP_SNOOP_FLAG when invoking Modify for snoop flows.

Even though there is a check to modify snoop only when snoopId is NOT DISABLED, still end up sending the modFlags as it is to PNPS, resulting in PNPS errors for modifying invalid snoopId.

Steps to Replicate: Issue was found during load testing with SIPREC enabled.

Workaround: N/A

There are unnecessary alarms on media port.

Impact: The SBC 5400 experienced false alarms for unused packet interfaces.

Root Cause: Code was missing to avoid unnecessary alarms for the unused packet ports.

Steps to Replicate: On an SBC 5400, disconnect the packet port cables and remove the configuration from the IpInterfacegroup to verify that alarms are not generated for unused packet ports.

The code is modified to handle cases where the packet port is not connected/configured in the IpInterfaceGroup to avoid unnecessary alarms.

Workaround: Connect the unused Ethernet ports to avoid getting alarms.

An SBC Memory Leak occurred.

Impact: When the inbound calls to the SBC are released early, the SCM Process leaks memory.

Root Cause: When the inbound calls are released early, the SCM Process does not release the memory allocated to store the SIP PDU.

Steps to Replicate: 

1. Change the mode of the ingress TG to out-of-service.

2. Send a high number of calls (10000+) to the ingress TG with 1 cps.
   All calls are rejected.

3. Verify the issue: Check the memory of ScmP. Memory has leaked.

The code is modified to ensure the SCM Process releases memory after early attempt call fails.

Workaround: Run the fix configuration which is causing early attempt failure.

Transcode percentage required to load DSPs in sweTrafficProfile.

Impact: On specifying 100% tonesPercent in custom profile without selecting any transcode percentage in swe Traffic profile, tpads are not loaded and dspStatus does not show any tones resource available.

Root Cause: This is due to an incorrect check that considers only the transcode percentage to allocate the DSP resource in custom profile activation script.

Steps to Replicate: 

1. Create a custom profile.
2. Allocate tone percent as 100.
3. Load the profile.
4. Check for show status system dspStatus.

This shows no toneAvailable.

Consider the transcode as well as tones percentage in the SWe traffic profile for allocating the DSP resources in custom profile activation script.

Workaround: Allocate some of the transcodePercent in the custom profile.

Portfix SBX-101188 (Originated in Release 9.1.0): The SBC halts when the disconnectSignalSequenceProfile is configured.

Impact: Configuring the disconnectSignalSequenceProfile in ASAN build causes the SBC to halt.

Root Cause: A stack-buffer-overflow occurs when accessing wrong data type in NrmaDiscSigSeqProfileSsCreate,

Steps to Replicate: Configure the disconnectSignalSequenceProfile in the latest ASAN build, and SBC should not halt,

The issue is fixed by changing the the data type from UCHAR to int32 in NrmaDiscSigSeqProfileSsCreate.

Workaround: N/A

Portfix SBX-101229 (Originated in Release 9.1.0): There was a stack-buffer-overflow in SipsGetSmmProfileForDlgScopehashUpdate.

Impact: There was a stack buffer overflow when 487 response is triggered toward ingress leg.

Root Cause: A stack buffer overflow occurs when accessing the freed memory in stack for hSipMsgHandle->pstlocalTsap.

Steps to Replicate: Repeat the CANCEL call scenario to reproduce the issue.

The code is modified to populate the right memory in hSipMsgHandle→pstlocalTsap.

Workaround: N/A

There was a SCM Process memory leak.

Impact: The standby SBC is leaking a per call structure that is used for Relay calls.
This leak can carry over to active when there is a switchover.

Root Cause: The leak is occurring because the code is overwriting the pointer to the structure, thereby preventing this structure from being freed when the call is completed.

Steps to Replicate: Upstream forking with INVITE and OOD messages

The code is modified to free the structure before the field that stores a pointer to it is overwritten.

Workaround: N/A

Observed a SCM process memory leak on the SBC 7000 platform.

Impact: A memory leak is observed when there are multiple calls to the function SipRaCopyAOR in the same call, the issue is observed during the call.

Root Cause: The existing memory is not freed before copying the new info to the endPointAor by using the SipRaCopyAOR.

Steps to Replicate: This issue is observed during the registration and call load scenario.

Run some registrations and make a call to registered users and run some load.

The code is modified to freed the existing memory of the buffer before copying the information into sipEndPointAor structure

Workaround: N/A

The CLI import's fail when uploading a CLI text file in the unix format.

Impact: The CLI import's fail when uploading a CLI text file in the unix format.

Root Cause: Removing the last character of each CLI line, which was making this issue.

Steps to Replicate: 

1. Navigate to Administration > System Administration > File Upload.
2. Click "Add Files to Queue", select CLI file to be import, and click "Upload All Files".
3. Navigate to "Configuration Script and Template Import".
4. Select .cli file from "Configurations" file list and click "Start Import".
5. See "Script and Template Status Monitor".

The code is modified to avoid the deletion of the last character in each CLI line that is read from the CLI file.

Workaround: N/A

The SBC has the same issue as SBX-86420. The SBC does not send the 'srv' query to DNS when the egress peer is FQDN and egress transport preference is TLS for outgoing REGISTER messages.

Impact: When TLS port is not sent from PSX, the SBC is not sending DNS srv query to DNS server. 

Root Cause: The root cause was when the SBC increments the port number by 1 and was sent by the PSX even if peer port was zero.

Steps to Replicate: 

1. Configure the SBC and PSX to send register and subscribe sip request to relay .
2. Configure the Serve FQDN and zero for the FQDN port.
3. Send a register/subscribe request.
4. The SBC should send SRV query to the DNS server.

To fix the issue, if port number sent by the PSX is zero then do not increment it by 1 by default for selecting the TLS port.

Workaround: N/A

Portfix SBX-102081 (Originated in Release 7.2.4): During the RTP-VTP 10 CPS, a 100 CHT G729 passthrough load was found on the CPU Congestion and CPU Spike multiple times.

Impact: There was intermittent CPU congestion reported for two vcpus overnight run.

Root Cause: When using 2 vcpus, there is no dedicated SIG core with both the MGMT core and signaling cores are shared, and the spikes in mgmt threads are causing the SIG congestion.

Steps to Replicate: Run two vcpu over night passthrough load.

The code is modified to disable the CPU congestion monitoring for < 3 vcpu, as it does not have any detrimental effect.

The momentary spikes of the MGT processes does not cause call failures, but raises false alarms of the CPU congestion.

Workaround: N/A

The Update (recvonly or inactive or sendonly) and Reinvite (recvonly) should not be converted to the CPG on the SIP-I side.

Impact: The ISUP standard does not allow the callee to put call on hold before call is answered.
The SBC sends an UDPATE message before answer with MIME body with CPG parameter with out hold or retrieve indication. Certain endpoints do not accept such SIP message and send 400 back to the SBC.
The second issue is the SBC sending re-INVITE to ingress with MIME body with CPG parameter but no hold or retrieve indication. Certain endpoints do not accept, such re=INVITE message and send 400 back to SBC.

Root Cause: The SBC sends SIP message with MIME body with a CPG parameter, but no hold or retrieve indication.

Steps to Replicate: Send an Update (call hold) before call answer or Reinvite (recvonly) after call answer from egress SIP side and check messages on ingress SIP-I side.

The code is modified to ensure the SIP message sent by the SBC to SIP-I endpoint are correct. The SBC does not send UPDATE and re-INVITE with MIME body without hold or retrieve indication. The SBC sends re-INVITE/UPDATE with SDP without ISUP body in such scenarios after the fix.

Workaround: Use a SMM workaround.

The SBC is sending unwanted UPDATE towards EGRESS leg with all the codecs.

Impact: The SBC is sending unwanted UPDATE towards EGRESS leg with all the codecs.

Root Cause: When the SBC receives a 200 OK(for UPDATE) from the UAC, the internal feature flag is not being set that causes MODIFY OFFER to be triggered, which causes the SBC to trigger UPDATE towards egress.

Steps to Replicate: 

1. UAC sent AMR-WB, AMR, PCMA as Offer to the SBC.
2. The SBC offered AMR-WB, AMR, PCMA to UAS.
3. UAS sent 180 Ring without SDP to the SBC.
4. The SBC started playing tone and sent 180 with SDP with AMR-WB.
5. UAS sent 183 with SDP PCMA.
6. The SBC sent 183 without SDP to UAC.
7. The SBC sent UPDATE with SDP PCMA.
8. UAC sent 200 OK (for UPDATE) with SDP PCMA to the SBC.

At this stage, the 200 OK(for UPDATE) from UAC, triggering a MODIFY OFFER towards NRMA causing the SBC to trigger an UPDATE towards the UAS that is not expected.

The code is modified so that, when 200 OK (for UPDATE) SDP is received from the UAC, the SG minimizes the media changes if there is no considerable SDP parameters change.

Workaround: N/A

The SBC does not generate the sonusSbxNodeResEmerCallNoRouteNotification.

Impact: The SBC does not generate the sonusSbxNodeResEmerCallNoRouteNotification alarm when the PSX returns no route for 911 based call (sip code 404, cause code 150).

Root Cause: The code to generate the sonusSbxNodeResEmerCallNoRouteNotification was not present in the scenario where the PSX returns no routes for the 911 based call.

Steps to Replicate: 

1. Make a basic SIP call using the PSX with no routes set for number starting with '911' on the PSX.
2. Configure the SBC:
   
   set profiles services emergencyCallProfile EmergencyCalls prefix 911
   
   set addressContext ADDR_CONTEXT_1 zone ZONE2 sipTrunkGroup SBXSUS12_LABSIP1 services emergencyCallProfile EmergencyCalls

The SBC does not generate the sonusSbxNodeResEmerCallNoRouteNotification alarm.

The code is modified to generate a sonusSbxNodeResEmerCallNoRouteNotification alarm when the PSX returns a "No Route" for emergency based call on matching URI prefix.

Workaround: N/A

Portfix SBX-102617 (Originated in Release 7.2.5): The SIPFE crashed causing corruption on the msgObject.

Impact: Using the SNMP to query ipPeer statistics may core if an invalid zone or ipPeer name is entered.

Root Cause: When there was an SNMP walk for invalid zone/ippeer, the SBC may double free memory (introduced by SBX-51006).

Steps to Replicate: Using snmptool to walk through the data:
snmpwalk -c admin -t 20 -v 2c -m all sbxsus5-1 1.3.6.1.4.1.2879.2.10.2.243.1.2.14.65.68.68.82.95.67.      79.78.84.69.88.84.95.49.5.90.79.78.69.50.4.84.78.84.49

The code is modified to avoid freeing memory twice.

Workaround: Ensure the valid zone/ippeer is entered when querying

Portfix SBX-94207 (Originated in Release 9.0.0): The 2 MSBC active setups did not come up after the configuration restore through the OAM in 3:1 MSBC setup.

Impact: Multiple expected reboots queued up so that the reboot count increased so much, that the service was not starting.

Root Cause: When the multiple reboots or/and restarts the queue up and the limit count is reached the service will not start.

Steps to Replicate: Perform a split brain, restore the configuration and unit tests.

Decrease the reboot and restart then count marker by 1, when the cause of reboot is known.

Workaround: N/A

Portfix SBX-102225 (Originated in Release 9.1.0) The SBCs fail to terminate call upon the receipt of BYE from MRFP due to the rtp inactivity timeout.

Impact: The SBC fails to terminate the call if the disconnection is received from the MRF for a session where the tone has been played and egress peer has performed mid-call modification.

Root Cause: In this call flow, after receiving a disconnect from MRF, the SBC wrongly picked up the tone context instead of the MRF context resulting in failure to terminate the call.

Steps to Replicate:

1. Early answer in 18x resulting in MRF transcoded call.
2. UAS sends 180 resulting in the SBC playing tones.
3. UAS sends SIP UPDATE to perform mid-call modification to change the codec.
4. MRF has been configured for RTP inactivity detection. Due to some reason, it does not receive any RTP packets. MRF sends BYE to the SBC.

Expected Result:
The SBC terminates the call gracefully after receiving BYE from MRF.

Actual Result:
The SBC is not terminating the call.

The code is modified to pick the correct context holding the MRF resource.

Workaround: None.

The call load along with switchovers and provisioning coredumps.

Impact: The IPsec data is stored for all signalling ports. The ipsec state array size was different from signalling ports array. Due to this issue, the ipsec state array is being overwritten.

Root Cause: Overwriting an array beyond its size led to memory corruption.

Steps to Replicate: 

1. While configuring the SBC, add a sigPort with index 4096.
2. Load testing at 500 cps for 15 hrs.
3. Run a switchover testing.
4. Physical port redundancy testing during load.
5. Customer configuration testing during load.

The crash should not happen.

The code is modified so that it holds the same number of entries as signalling Ports array.

Workaround: While adding sigPorts, do not add sigPort index 4096.

The LRBT had an unexpected UPDATE with the AMR-WB codec sent.

Impact: The SBC is not playing the tone using the same codec as negotiated, when the 183 with SDP is received and the same is indicated to ingress with UPDATE.

Root Cause:  It is a race condition exposed under the following conditions:

1. A calls B and B sends a 180 without SDP causing the tone to be played to A.
2. Subsequently B sends 183 with SDP so that the SBC needs to send UPDATE out to A for changing the codec.
3. Soon after the 183 with SDP from ingress is received, a 180 without SDP causes the tone play to start.
4. The race condition can be visualized as the UPDATE's 200 OK with SDP trying to modify the system for CUT-THRU, but instead interferes with resource management subsystem's attempt to play tone.
5. As a result, the resource management subsystem picks the incorrect PSP's to play tone.

Steps to Replicate: This issue is not easily reproducible and requires a set of configuration that customer has:

1. No DLRBT.
2. No Downstream forking.
3. The Ingress has AMR - WB and NB codec and some other NB codec.
4. The Egress has only NB codec's.
5. Transcode is not allowed b/w WB and NB codecs.

The code is modified to enforce that while picking the PSP's to modify when TONE is on, pick the latest PSP and not an earlier one.

Workaround: Providing for Transcode b/w WB-NB will a correct call.

The flag 'statusUpdateSupport' is not working.

Impact: The SBC includes the Accept and Allow headers while generating OPTIONS ping requests towards the peer even when the statusUpdateSupport flag is disabled.

Root Cause: The code was not setting the flag correctly and passed the "Accept" and "Allow" headers irrespective of the statusUpdateSupport flag.

Steps to Replicate: To replicate/verify the issue configure the path check profile and set the stausUpdateSupport flag disabled. The SBC does not send the Accept and Allow header in the OPTIONS ping towards the peer.

The code is modified to handle the statusUpdateSupport flag correctly.

Workaround: The customer can use the SMM rule to remove the Allow and Accept headers from the OPTIONS ping.

Portfix SBX-102767 (Originated in Release 9.1.0): A call from the subscriber number was not allowed after multiple switchover/standalone restart.

Impact: Configured the subscriber list for a surrogate Aor is not restored if the SBC is restarted/Switchover/LSWU.
So a call from a configured subscriber for a surrogates Aor will fail.

Root Cause: When restoring the IpPeer configuration from shared memory as part of SBX-62864 (Optimizing config load time), the restoration surrogate subscribe was missed from the SIPFE module. Optimizing the configuration load time feature had very big changes.

Steps to Replicate: 

1. Configure Surrogate Registration Profile and Aor with configured Subscriber list.
2. Attach it to an IpPeer and enable surrogate registration state at Profile and IpPeer.
3. Make call for a configured subscriber call will be allowed and 401 challenge will be authenticated.
4. Restart the SBC.
5. Make call for a configured subscriber call will be allowed and 401 challenge will be not be authenticated.

The code is modified so the surrogate Registration AOR subscription list is restored to the SBC restart/Switchover/LSWU.

Workaround: Perform a SBC restart/Switchover/LSWU disable and re-enable surrogate state at IpPeer.

The SCM process coredumps SipSgHashLookup on the OOD OPTIONS/Subscribe.

Impact: The SCM process may coredump while doing a SipSgHashLookup on a OOD OPTIONS/Subscribe message.

Root Cause: The sipsgRelayCbHashTbl was corrupted where a list head is corrupted and there is not additional information to state what caused the corruption. The issue may have been caused by an entry added to the hash table twice (maybe with different keys).

Steps to Replicate: Not reproducible in the lab. 

The code is modified to ensure the hash table entry is explicitly removed from the hash table, even if the hashlookup fails and also avoids adding a duplicate entry.

Workaround: N/A

Portfix SBX-98602 (Originated in Release 9.0.0): The AddressSanitizer detected the heap-use-after-free on the address 0x61100006da70.

Impact: When the SBC relayed messages (e.g. UPDATE, REFER etc.), it read a memory block after it was freed.

Root Cause: The code that held a pointer to location inside a larger memory block, the larger memory block got freed but then immediately afterwards the code was doing a read to check a value from the pointer.

Steps to Replicate: This issue is only highlighted when running with ASAN enabled images in the engineering lab.

The code is modified so that it no longer accesses memory after it is freed.

Workaround: N/A

Portfix SBX-96729 (Originated in Release 9.0.0): The AddressSanitizer detected the heap-use-after-free on the address 0x6180000bd500 while running the IPSECAKA-UDP suite.

Impact: While deleting the IPSEC configuration, the SBC was reading memory after it had been freed up.

Root Cause: There was some debug code that was printing information from the memory block, even after it had been freed up.

Steps to Replicate: This is problem is only highlighted when we run IPSEC delete configuration commands in the engineering lab with ASAN enabled images.

The code is modified not to read the memory contents after the contents are freed up.

Workaround: Do not delete the IPSEC configuration.

Portfix SBX-96427 (Originated in Release 9.0.0): The AddressSanitizer detected a alloc-dealloc-mismatch (operator new vs free) on 0x603002288530.

Impact: As part of security related testing, when the memory was getting freed, instead of using delete the SBC code was using free to release the allocated memory.

Root Cause: In the SBC code, when the memory was allocated, the code used was new but during deallocation, it was using free instead of delete.

Steps to Replicate: This problem is only highlighted when running with ASAN enabled images in the engineering lab.

The code is modified to use delete for memory deallocation instead of free.

Workaround: N/A

Portfix SBX-101401 (Originated in Release 8.2.2) The call disconnected when the 10 PSX routes are returned.

Impact: Run an Invite Call Flow with the customer configuration where the PSX returns 10 Routes in the call, the SBC is disconnecting the call.

Root Cause: The IcmParamInsert is failing for one of the paramtype in the NrmaCcSelectEgressSgCmd as the size is exceeding max size ( ICM_REQUEST_MAX_12)

Steps to Replicate: Run a call flow with the customer configuration where the PSX returns 10 Routes per call.

The maximum size is increased to ICM_REQUEST_MAX_14 to address the issue.

Workaround: N/A

Portfix SBX-102927 (Originated in Release 9.1.0) The EVS codec attributes are non-readable after an upgrade from 7.2 to 8.2.

Impact: The upgrade fails to set the default value for the maxChannel flag.

Root Cause: The upgrade fails to set the default value for the maxChannel flag.

Steps to Replicate: 

1. Create the codec profile in with codec=evs in sbc<8.1.
2. Upgrade the SBC.
3. The maxChannels value should be shown in CLI as 1.

The code is modified to set the default values during an upgrade.

Workaround: The value in db for attribute3 in code entry table should be modified to set the default value of maxChannels.

Some counts in callCurrentStatistics keep incrementing.

Impact: The "activeRegs" counter provided by the CLI zone callCurrentStatistics / callIntervalStatistics command may continuously increase.

Root Cause: There are badly formed SIP REGISTER messages received by the SBC that will increment the "activeRegs" counter and never decrement it.

Steps to Replicate: Send bad SIP REGISTER message(s) to the SBC, and see that the "activeRegs" counter provided by the CLI zone callCurrentStatistics / callIntervalStatistics command increases (and does not decrease).

Decrement the "activeRegs" counter when the SIP REGISTER message fails the SIP parser to address the issue.

Workaround: None.

The SCM process coredumped.

Impact: The SCM process may coredump during the gateway to gateway calls using SDP transparency.

Root Cause: The software packed and unpacked unsupported content header types causing NULL pointer access.

Steps to Replicate: 

1. Configure the SBC for gateway to gateway calls using an external PSX.
2. Configure the transparencyProfile on both ingress and egress trunk groups enabling sipMessageBody all.
3. Configure the direct media and SDP transparency on both ingress and egress trunk groups.
4. Perform SIP gateway to gateway call flow: INVITE, 183 with SDP, PRACK, 200 (prack), 180 with SDP, PRACK, 200 (prack), 200 with SDP, ACK, BYE, 200.

The code is modified to prevent the unsupported content header types from being packed and unpacked.

Workaround: The sdpTransparency is not supported over the gateway to gateway, disable the signaling sdpTransparency sdpTransparencyState.

Portfix SBX-102495 (Originated in Release 7.2.1R10) The authentication process is not performed when calling for OTT.

Impact: New calls from child subscriber numbers of a particular AOR would fail after deletion of one or more subscriber numbers from the same AOR.

Example: profiles->services->surrogateRegistrationProfile->aorUserName->subscriberNumbers has a set of 10 numbers. If one or more numbers are deleted from this list (but the entire list is not deleted), then new calls from the remaining numbers in the list for the particular AOR user name fails.

Root Cause:  The ConfD version was upgraded from 6.4 to 6.7 in the 7.2.0R0 release.

In the ConfD version 6.4, the addition or deletion of elements from subscriberNumbers list was being notified to the SBC application as MOP_VALUE_SET and the SBC was invoking the function to handle modification to the list. Deleting the entire list was notified as MOP_DELETED and the SBC would invoke the function to handle deletion of the entire list.

In the Confd version 6.5 and above, the addition of elements to subscriberNumbers list gets notified as MOP_CREATED and the deletion of elements gets notified to the SBC application as MOP_DELETED. So after upgrading to 7.2.1R9, when the user deleted a few elements from the list, the SBC was notified with MOP_DELETED and it invoked the function to handle deletion of entire list (as it used to do earlier).

Steps to Replicate: 

1. Create a surrogate registration profile with the AOR and child subscriber numbers.
2. Validate that call from a child subscriber number is working upon successful registration of the AOR.
3. Delete a subscriber number from the AOR.
4. Validate that call from the deleted subscriber number is not authenticated upon receiving challenge from the proxy.
5. Validate that calls from remaining subscriber numbers are validated/authenticated by the SBC upon receiving challenge response from proxy.

The code is modified to handle MOP_CREATED and MOP_DELETED notifications from ConfD to handle cases of creation,modification and deletion to the subscriberNumbers list.

Limitation note: Multiple subscriber numbers can be provided in a single command but any subscriber list related command needs to be part of a separate commit* (There cannot be two subscriber list commands in a single commit)

Workaround: Delete the entire subscriberNumbers list:

delete profiles services surrogateRegistrationProfile <profName> aorUserName <aorName> subscriberNumbers
commit

Add back the subscriber numbers needed:
set profiles services surrogateRegistrationProfile <profName> aorUserName <aorName> subscriberNumbers <num1> <num2> <num3>
commit

The trace file containing a SIP Rec PDU was not imported in Ribbon Protect properly.

Impact: When level 4 call trace is active, if a SIP PDU is too large to fit on a single trace line, it is split over multiple lines. However, Ribbon Protect only reads the first line as there is no way for it to know subsequent lines are continuation lines.

Root Cause: The root cause came from a design issue.

Steps to Replicate: 

1. Enable level 4 call trace.
2. Send a SIP PDU to be traced towards the SBC of total size 2000 bytes.

The code is modified to put the same header onto each level 4 trace line. Include in the header two new fields, MSG ID and PART to allow Ribbon Protect to recombine multiple trace lines to recover the original message. The equivalent Jira VIGIL-17137 is required for Ribbon Protect compatibility.

Workaround: None.

Portfix SBX-101237 (Originated in Release 9.1.0) Incorrect callMediaStats for rfc2833-rfc2833 passthru call on the SBC.

Impact: There was an incorrect callMediaStats for rfc2833-rfc2833 passthru call on the SBC.

Root Cause: For PassThru calls, when multiple codecs are configured on both legs in "Codecs Allowed For Transcoding", Active PSP on ingress leg does not have fully qualified selectedAudioEncode. As a result, ingress dtmfmode value contains incorrect value.

Steps to Replicate: 

1. Make a sip call A-B with codec G723 with DTMF relay set to RFC2833.
2. Configue DTMF preferred payload type to 101 in both egress and ingress PSP.
3. Enable multiple codecs including G723 on both legs in "Codecs Allowed For Transcoding" in the ingress and egress PSP.

The code is modified to use the matching codec entry from audioEncode array instead of the selectedAudioEncode, and as a result takes the right value for ingress dtmfmode of callMediaStatus.

Workaround: None.

Portfix SBX-102906 (Originated in Release 9.1.0) The LSWU from 6.2.1R0_2 to 9.1.0R0_8904 fails.

Impact: An upgrade failed due to an unsupported timezone.

Root Cause: The customer timezones are no longer supported, but the upgrade of these timezones were not handled.

Steps to Replicate:

1. Configure one of the three customer timezones.
2. Do an upgrade to 9.1R0(with fix).
3. The upgrade should succeed.

The code is modified to support the customer timezones. After an upgrade, the customer's three timezones are upgraded to Asia/Riyadh.

Workaround: None.

SBX-102992 | SBX-103001

The SBC is blacklisting a peer when the mid dialog request gets timed out even though "midDialogArsScreenLevel" flag is set to "never" in sipArsProfile.

Impact: The SBC is blacklisting peer when a mid dialog request gets timed out even though the midDialogArsScreenLevel flag is set to never in sipArsProfile.

Root Cause: The original design only covered original INVITE messages and not mid-call INVITE messages.

Steps to Replicate:

1. Make a call from UAC.
2. Answer the call on UAS.
3. When the call is stable send a Re-INVITE request from UAC.
4. The SBC forwards this Re-INVITE request to UAS.
5. Do not send any response to this Re-INVITE from UAS.
6. The SBC sends BYE and terminates the call.
7. Check the "sipArsStatus" on egress zone.

The code is modified to correctly handle mid-dialogue INVITE's as well as original INVITEs.

Workaround: None.

Portfix SBX-102976 (Originated in Release 9.1.0) The diam process crashed while doing Rf specific configurations.

Impact: The "revert" option should be used before proceeding when there is an error returned from the confd. The DiamProcess coredumps when diamNode is configured incorrectly and proceeded without "revert" option.

Root Cause: A missing NULL check is the root cause for this issue.

Steps to Replicate: None.

Added NULL check to fix the issue.

Workaround: The "revert" option can be used when the confd throws an error and then proceeds with the further configuration.

Portfix SBX-91016 (Originated in Release 9.1.0): Unable to see signaling messages on the PKT log file.

Impact: The packet capture does not work properly for PKT ports

Root Cause: version of libpcap0.8 library in Debian9 has a defect which results in the capture not working

Steps to Replicate: 

1. Login to SBC and do basic call config.
2. Create a signaling packet capture filter: set global callTrace signalingPacketCapture state enable devices pkt0 0.
3. Run SIPP calls.

Expected Results:

1. The SBC should capture all the signaling messages on the pkt0 port in a PKT file under gsxlog directory.

The code is modified to a newer version to address the problem.

Workaround: None.

The SBC does not transfer contents of /var/log/syslog to the remote server whenever the log file is rotated.

Impact: The syslogLog, sftpLog, consoleLog, ntpLog and platformAuditLog fail to send logs through the syslog, when file sizes are decreased (for example when logs are rotated).

Root Cause: If a file size decreases but inode stays the same, then rsyslog does not (by default) know that the there is new logs being written to the files, therefore will not send these new logs.

Steps to Replicate: 

1. Setup logging to remote server for log type.
2. Run logrotate: logrotate -f.
3. Verify if new logs are written to the remote server.

The code is modified to reopen a file if it detects the the file was truncated.

Workaround: Edit /etc/rsyslog.conf and add reopenOnTruncate="on" to all input() rules.

For example:

#input(type="imfile" File="/var/log/syslog" Tag="syslog" Severity="info" Facility="news" reopenOnTruncate="on")

Portfix SBX-93790 (Originated in Release 9.2.0) The TEAMS user not able to resume the call when transfer is failed, when the call is initiated in beginning by PSTN user.

Impact: During a Refer call, if the transfer target sends early answer in 18x but then rejects the call, the SBC fails to resumes the previously active call.

Root Cause: During call transfer, after tone play, while processing early answer in 183, SBC wrongly freed the previous cut-thru context and instead retains the previously activated tone context (for A-B call).

As a result, after transfer target rejects the call, the SBC attempts to resumes the previously active call (A-B) that fails due to unavailability of correct context.

Steps to Replicate: 

1. PSTN1 to TEAMS call.
   TEAMS transfer call to PSTN2.
   PSTN2 rejects the call.
   TEAMS resume the call.
2. PSTN1 To TEAMS call.
   TEAMS transfer call to PSTN2.
   PSTN2 does not answer the call.
   TEAMS resume the call.

The code is modified to retain the previously activated cut-thru context and free the previously activated tone context if current tone context is more recent.

Workaround: N/A

The SIPFE sg allocation calls that always land on the SCM0 until exhausted.

Impact: A call from the AS without a registration may always land on the SCM0.

Root Cause: The SBC accidentally selected prefer SCM0 when no registration was found.

Steps to Replicate: Configure the zone xxx remoteDeviceType appServer, incoming call to zone xxx and without registration. The call will route to the SCM0 until the system is exhausted.

If there is no registration, the SBC round robins the SCMs.

Workaround: N/A

Portfix SBX-102470 (Originated in Release 9.1.0) SAM Process memory leak while running the TLS/SRTP load on the Openstack I-SBC.

Impact: The SIP-TLS with an Client Authentication (authClient=true in tlsProfle) causes about 1.2 KB of memory leak per TLS handshake on the SBC.

Root Cause: After verifying the peer certificate, the SBC SAM process did not free memory allocated for accessing the public key.

Steps to Replicate:

1. Run 50,000 SIP-TLS sessions with Client Authentication, and observe the memory used by SamProcess.
2. Run another 100,000 SIP-TLS sessions with Client Authentication, and observe the memory used by SamProcess.
3. Compute the extra memory used for additional 100,000 SIP-TLS sessions.

The code is modified to free the memory allocated for accessing public key after its use.

Workaround: None.

Portfix SBX-102747 (Originating in Release 9.1.0) The USAGE_LIMIT and IN_USE columns of SystemLicenseInfoStats are updated with double the actual value of all the features in the licenseInfo table.

Impact: After a cloud instance running in Nto1 switches over, the USAGE_LIMIT and IN_USE columns of the SystemLicenseInfoStats are updated with double the actual value of all the features in licenseInfo table.

Root Cause: The code was not correctly updating the list of SBC processes that are contacted to obtain stats.

Steps to Replicate:

1. Run an N to 1 instance with at least 2 nodes.
2. Failover the active.
3. Check that the license SystemLicenseInfoStats pms file does not contain double the number in the USAGE_LIMIT and INUSE columns that is reported in system licenseInfo.

The code is modified to properly update the list of SBC processes that are contacted to obtained stats.

Workaround: After the former active restarts, restart it again.

Portfix SBX-102932 (Originated in Release 9.1.0) When the Egress leg is tapped, a request-uri parameter is not sent in the SIPREC Metadata.

Impact: The request-URI is not added into the callData section of metadata sent towards the SRS while tapping the egress leg, though it is configured in SipRecMetaDataProfile.

Root Cause: This is day one issue where the RequestURI addition was not handled while tapping the egress leg.

Steps to Replicate: Make A to B call.

The code is modified to handle the requestURI while tapping the egress leg.

Workaround: No workaround.

Portfix SBX-103005 (Originated in Release 9.1.0) The RequestURI header content is not correct in SIPREC Metadata profile.

Impact: The request URI header populates in the metadata is not added the URI scheme.

Root Cause: This scenario was not handled from day one.

Steps to Replicate: Make A to B call.

The code is modified to fix the issue.

Workaround: None.

Portfix SBX-103004 (Originated in Release 9.1.0) The Content-Length header is missing for the SIPREC Metadata message body.

Impact: The content-length header is missing for the SIPREC Metadata message body.

Root Cause: This was not supported from day one.

Steps to Replicate: Make A to B call.

The code is modified in the SIPREC Metadata message body to address the issue.

Workaround: No workaround.

The configuration flag "Sdp100relIwkForPrack" behavior is incorrect.

Impact: In a latemedia passthrough call, the SBC is not sending ACK for 200 OK when an Asymmetric PRACK Interworking feature is used.

Root Cause: The SBC fails to relay 200 OK for UPDATE in late media passthrough and a reverse offer scenario. This issue is fixed but the given fix breaks the Asymmetric PRACK feature functionality.

Steps to Replicate: Configuration:
1. Set the flag lateMediaSupport to passthru on the ingress TG.
2. Enable the 100rel Support on the ingress TG.
3. Enable the flag Sdp100relIwkForPrack on the egress TG.

Procedure:
UAC sends the Initial INTIVE request to SBC without sdp and has 100rel in Require header
UAS sends the 180 towards SBC with no sdp after receiving offer less Invite.
UAC sends PRACK with SDP answer after receiving 180 with SDP offer.
UAS sends the 200 OK towards SBC with sdp offer.
UAC sends ACK after receiving 200 OK.

Expected Result:
The SBC should send ACK with SDP answer on the egress side.
After re-negotiation media communication should be done as per final offer answer communication

The code is modified to address the issue.

Workaround: None.

Portfix SBX-101876 (Originated in the Release 9.1.0) Possible NRS ICM message leaks.

Impact: There is a small memory leak when adding and deleting ACL rules.

Root Cause: The internal ICM message that is used to trigger the addition and removal of ACL rules in the NRS subsystem is not being freed up.

Steps to Replicate: Add and remove ACL rules and check for memory increasing.

The code is modified to correctly free the ICM memory block after it is processed to avoid the memory leak.

Workaround: None.

Portfix SBX-103006 (Originated in the Release 9.1.0) The SIPREC Metadata schema has wrong value for the nameID AOR tag.

Impact: The nameId AOR formed in the metadata is incorrect when the P-Preferred-Identity header with alphabets in the UserName is received.

Root Cause: Username of P-Preferred-Identity header is modified internally based on E164 format.
P-Preferred-Identity is copied for SIPREC metadata purpose after the above modification. As a result, this caused the issue.
NOTE: The P-Preferred-Identity header with the numeric UserName is handled already.

Steps to Replicate: Make a A-B call with the P-Preferred-Identity header sent from A party and A party is recorded.

The code is modified for SIPREC metadata purpose before the E164 modification.

Workaround: No workaround.

Portfix SBX-103137 The Ingress SBC is tearing the call down by sending unexpected BYE to the ingress endpoint.

Impact: The HDCodecPrefered flag is not working as expected if the SDP contains the first two codecs as HD.

Root Cause: If the SDP has first two consecutive HD codecs and HDCodecPreferred flag is enabled, the SBC is incorrectly considering the second HD Codec as NB.
This results in the SBC treating a HD codec as NB while applying the HD rules.

Steps to Replicate: The HD Codec preferred flag enabled
The offer contains multiple HD codecs (the first two being HD)

1. Create the Codec entries with codec G722,G711,G722.1,G729.
2. Configure PSP HDPSP1 with codecs (G722,G711,G729,G722.1), gw PSP GWHDPSP with codecs (G722.1,G711,G729,G722) and HDPSP2 with codecs (G711,G722,G722.1,G729).
3. Enable the following flags:
a) 'HDCodecPreferred and SRPP' flags in gw PSP.
b) 'All Codecs Allowed For Transcoding' in PSPs.
4. Make a LRBT call in which UAC sends INVITE with G722,G722.1.
5. UAS reply '180' without SDP
6. UAS reply '200 OK' with G711.

Expected:
1. Egress side INVITE must have codecs as per the sequence below, G722.1,G722,G711,G729.
2. Ingress side '200 OK' must have codecs as per the sequence below,
G722.
3. A call should be successful with G722 to G711 transcode on ingress side and G711 passthrough on egress side.

Do not tear down the call, due to the SBC treating the second HDCodec as NB.

The code is modified to consider it as a HD codec.

Workaround: None.

There was an error on a SBC application switchover.

Impact: When an emergency call is terminated with a delay and incoming TG is marked out of service, the SMC process may core.

Root Cause: When an emergency call is terminated with a delay and incoming TG is marked out of service, the SMC process may core.

Steps to Replicate: Run an emergency call and ensure the SBC does not core.

The code is modified to ensure the SIP service group pointer is checked before de-referencing.

Workaround: N/A

An error occurred during the CLI input being executed in the customer SBC.

Impact: The SM process cored when the CDB is loaded with bulk configuration.

Root Cause: Running dmesg got blocked when bulk configuration was loaded into CDB leading to healthcheck failure.

Steps to Replicate: None.

Disable healthcheck when the dmesg command is running to address the issue.

Workaround: N/A

The diameter connState is closed after a switch-over.

Impact: Without this fix for Hardware and 1:1 deployments after a switch-over from the new Active, the diameter connection towards the diameter peers will not be initiated.

Root Cause: During a transition of the standby to active, one of the data types present in fetching the diameter peer from CDB is wrong. Due to this issue, the diameter peer configurations are not fetched from the CDB.

Steps to Replicate: 

1. In Hardware or 1:1 HA setup, establish diameter connection from the active SBC.
2. Perform a switch-over.
3. After the standby becomes active, connection towards the diameter peers are not initiated.

The code is modified so that the diameter peer configurations are fetched properly by the application.

Workaround: N/A

The network interfaces are not renamed as expected after an upgrade from 820R2 to 821R0.

Impact: The network interfaces are not properly mapped when the system is abruptly brought down after a first boot and rebooted.

Root Cause: Cloud-init service runs in parallel to sonusudev service and can cause issues with network interface mapping as it tries to rename the interfaces.

Steps to Replicate: Reboot system after sonusudev is run on first bootup.

The code is modified to start only after sonusudev is ran and mapped the interfaces properly.

Workaround: Reboot the instance again to recover from the issue.

The application restarted because of of a SCM process core.

Impact: For a NativeForking scenario, the SCM process may core while deactivating media resources.

Root Cause: Based on a core analysis for a NativeForking scenario, while trying to deactivate media resources, the call control block may trigger null pointer exception.

Steps to Replicate: Run basic call forking scenarios.

The code is modified to ensure the NRMA deactivation pointer is checked before de-referencing.

Workaround: N/A

Portfix SBX-100752 (Originating in Release 9.1.0) The "Filter Status" page stuck in loading while selecting a value from the list.

Impact: The "Filter Status" page stuck in loading while selecting a value from the list.

Root Cause: The reload icon is not removed after the loading the screen.

Steps to Replicate:

1. Log in to the EMA.
2. Navigate to Administration -> Accounting and Logs -> Event Log -> Filter Status.
3. Try selecting a value from the "Filter Status list". After clicking on the radio button, the page is stuck in loading phase.

The code is modified to remove the reload icon.

Workaround: None.

The SBC modifies the Replaces parameter with the wrong Call-ID and tags if the Replaced call was looped back to the SBC through a SIP Proxy that does not modify SIP Call-ID and tags.

Impact: Relay a refer with replaces for loopback call, the SBC picks up the wrong call leg.

Root Cause: The SBC query for the replaces callId, and found matching both legs (loopback).The SBC pick up the wrong leg.

Steps to Replicate: This is specific to customer call flow.

If loopback is detected, only pick the one with to-tag match local-tag. This is per RFC-3891, section 3.

Workaround: N/A

Calculated the size of a buffer as too small and error logs were being generated post upgrade.

Impact: Removes this incorrect log message:

SipsRedGetCallControlSizeCmd()
Minor .SIPSG: *SipsRedMirrorCallControlCmd: Potential for buffer
format ERROR. Available:16076 Calculated:383 Packed:277

Root Cause: This message is logged incorrectly when the code is calculating the size for a Redundancy message.

Steps to Replicate: The steps cannot be replicated.

The code is modified to remove this incorrect log message:

SipsRedGetCallControlSizeCmd()
Minor .SIPSG: *SipsRedMirrorCallControlCmd: Potential for buffer
format ERROR. Available:16076 Calculated:383 Packed:277.

Workaround: N/A

The SIPCM (SAM) deadlock reported on the SNMP request.

Impact: The SAM process cores when getTlsStatus command is executed.

Root Cause: The issue is because when fetching tls Status, Design was trying to access sessData that was no longer valid.

Steps to Replicate: Execute the getTlsStatus command when running TLS load. The SAM process cores randomly.

The code is modified to fetch the status from socketPtr instead of the sessData.

Workaround: N/A

The large microflow profile was not getting enabled in the Custom Traffic Profile.

Impact: 

1. For the default profile, max subs were always set to 256K.
2. For custom and standard traffic profiles, the micro flow count in NP resource was not correct.
3. limited support of 2M micro flows for standard profiles.

Root Cause: 

1. Large micro flow support was not there for custom traffic profiles.
2. For standard and default traffic profiles, there was restricted/incomplete support of large micro flow which is made generic as part of this JIRA.

Steps to Replicate: 

1. Create a instance with 10GB memory and 16vpcu with default traffic profile. Check microflow limits using "/opt/sonus/bin/cpsi -d summary" command.
2. Increase memory to 50GB memory and check microflow limits using "/opt/sonus/bin/cpsi -d summary" command.
3. Create a instance with 20GB memory and 16vpcu. Create and activate a custom traffic profile with access enabled in call mix. Check microflow limits using "/opt/sonus/bin/cpsi -d summary" command.

The code is modified to:

1. To introduce a new estimation parameter maxSubs based on which micro flow count is decided on and NP hugepages that are reserved.
2. To enable large micro flow support for all standard/default profiles where mem > 48GB and vcpu_count >10

Workaround: N/A

Portfix SBX-103273 (Originated in Release 9.2.0) There was dual NUMA support in the SWe.

Impact: Multiple NUMA were not allowed in general (except few cases).

Root Cause: Restriction was imposed not to allow multiple NUMA in HostCheck.

Steps to Replicate: 

1. Configure a KVM instance with dual NUMA and install 8.2.3R0 in Non-Gold/Non-GPU setup.
2. Let it come in default traffic profile.
3. Verify that the SBC would come up fine without any HostCheck errors.

The code is modified to allow multiple NUMA in general irrespective of personality and profile.

Workaround: N/A

When the PAI does not have a userinfo and when "Prefer PAI" is enabled, the verstat does not get added either to PAI/ From header.

Impact: When PAI header does not contain userinfo part, the verstat is not added to PAI or From header with the control set to "Prefer PAI".

Root Cause: While using the verstat as an user param, while formating the PAI header without an userinfo, the vetstat is dropped.

Steps to Replicate: Configure the STI profile, and send in an INVITE with no userinfo part in the PAI header. Configure PSX STI profile Verification control for verstat to "Prefer PAI".

The code is modified to add a verstat to the From header when none of the PAI header have a userinfo part and the control is set to Prefer PAI.

Workaround: N/A

Portfix SBX-97373 (Originated in Release 9.0.0) The SLB was establishing the connection on a IPv4 and IPv6 at the same time.

Impact: The SLB was establishing the connection on a IPv4 and IPv6 at the same time.

Root Cause: When the SLB commIntf is changed, the UMS server thread was being stopped and restarted with new interface address but the registration data of current clients (SBCs) registered to the SLB were not being deleted. So the SBCs were still showing up as "Connection State: up" in the sipClientStatus command.

Steps to Replicate: Change the SLB or SBC commInterface and ensure old connections are deleted.

The code is modified to initiate a deletion of CNode on Cluster Manager, Agent and sync it to standby. From the SBC side, calling SipFeSlbClientShutdown to delete client connections from the SBC when commIntf is changed and call SipFeUtilStartSlbClient.

Workaround: N/A

Portfix SBX-98695 (Originated in Release 9.0.0) There was a possible memory leak seen in the standby SLB.

Impact: A memory leak in standby SLB SCM process. There was a stale TCP connection seen in the active SLB when the SBC goes down.

Root Cause: Code was not present on the standby SLB to delete the data structure that stores the SBC details, when the SBC disconnects from the SLB. The TCP connection was not cleared correctly when the SBC disconnects from the SLB after SLB usage is disabled.

Steps to Replicate: 

1. Attach a valgrind to the SCM process on the standby SLB.
2. Once the SBC is connected to SLB, shut down the SBC.
3. Bring down the SLB too and check valgrind logs.

For TCP connection

1. Disable SLB usage and do sbxstop on the SBC.
2. Verify if all TCP connections between the SBC and SLB is deleted correctly.

The code is modified to delete the internal data structure on standby SLB when SBC disconnects from SLB.

Also clearing TCP connection when SLB usage is disabled because it may run a sbxrestart after disabling the SLB usage (so active TCP connection could be present).

Workaround: N/A

There is a new BMC/BIOS version for 8.2.3.

Impact: The BMC and BIOS versions are not up to date.

Root Cause: The BMC and BIOS versions are not up to date.

Steps to Replicate: Upgrade BMC before start app install

The code is modified so the BMC and BIOS firmware images are updated.
Firmware-5XX0-V03.22.00-R000.img
Firmware-5400-V03.22.00-R000.img
Firmware-7X00-V03.22.00-R000.img

Workaround: N/A

Changing the XRM to build ALL media RID’s static IP headers with the “Don’t Fragment” (DF) bit SET TO 0 instead of 1.

Impact: The SBC forwards the DTLS packets correctly but the large certificated packet does not reach the calling device.

Root Cause: The SBC has set the DF (do not fragment) bit in IPv4 header so routers cannot split the packet and it can result in the packet getting dropped.

Steps to Replicate: 

1. Setup a Inter-work between Full ICE client and ICE Lite call flow with DTLS/SRTP relay.
2. Caller send out large DTLS packets.
3. Capture the packets, ensure the DF bit has not set (0x0000).

The code is modified to set the DF bit when the SBC builds the IP header for DTLS and STUN packets.

Workaround: N/A

Portfix SBX-103883 (Originated in Release 9.2.0) A low transcode capacity in 2 VCPU.

Impact: 

1. The 2 VCPU SWe instance was not coming up in 8.2.3 release.
2. The transcode capacity in a 2 VCPU instance was very less, only 40-45 calls were supported.

Root Cause: 

1. There was a merge issue during 7.2.4->8.2.x merge, due to which one constant SESSIONS_SMALL_SWE was missing, so there was a script error and 2 VCPU was failing to come up.
2. Even though 2 vcpu uses default profile, the capacity of UXPAD is specially handled to have full UXPAD capacity instead of 1/4th of 1 UXPAD capacity.

Steps to Replicate: 

1. Bring up a 2 VCPU SWe instance in 8.2.3 release.
2. Check "show table system dspStatus" CLI command and verify available transcode resource. Make a load run till 90% of utilization of transcode channels and verify the packet stats ( pktloss/jitter).

The code is modified to have full capacity of a UXPAD process.

Workaround: No work around is available. Need to pick the build after the fix.

The code is modified to allow the message body QSIG Content-Type: multipart/mixed, Resource/Lists across GW-GW .

Workaround: N/A

EMS SBC Manager import of SMM remains "In Progress" even if it is completed.

Impact: EMS SBC Manager import of SMM remains "In Progress" even if it is completed.

Root Cause: When a user performs a Start Import operation from the Configuration script and the template import screen, and stays on the same screen, the SBC Manager makes a getStatus API call which reads the status (using platform manager getStatus API) and updates into a map object. If the user switches to another screen (other than the Configuration and profile import/export screen) there will be no issue, but if the operator switches to the Configuration and profile import/export screen which makes platform manager getStatus API call which does not update the a map object. After completion of start import, we read the status from the map and display to the UI.

Steps to Replicate: 

1. Log in to EMA.
2. Navigate to Administration -> System Administration -> File Upload.
3. Upload a CLI file.
4. Navigate to Administration -> System Administration -> Configuration script and template Import screen.
5. Select the uploaded the CLI file and click on start import.
6. Navigate to Configuration and Profile Import/Export screen.
7. Stay on that page until the START import is complete.
8. Navigate to Administration -> System Administration -> Configuration script and template Import screen
9. If the CLI executes successful or failed, we should be able to see the correct status.

The code is modified to make a difference in the EMA call or PM call.

Workaround: N/A

The MS Teams Carrier Trunk MOH fails to resume after a sixth hold.

Impact: The CAC is incorrectly double counting on a call pickup when the SMM invokes a move TG.

Root Cause: During an SMM move of a TG, the call pickup information is lost, resulting in double CAC counting.

Steps to Replicate: 

1. Run a TEAMS to PSTN call.
2. Run a TEAMS put call on hold (MOH) more than 6 times.
   

The code is modified so when the SMM moves a trunk, the call pickup information is moved along with it.

Workaround: None.

Portfix SBX-102714 (Originated in Release 9.1.0) 50% of calls are getting dropped after the SBC switchover.

Impact: The stable calls are getting dropped after an SBC switchover with GW-GW setup.

Root Cause: With the introduction of cloud, the slot ID selection was changed to 6th bit from 5th bit, but the GSX and SBC hardware and software still uses 5th bit for slot ID. This was affecting a stable call, when a switchover occurs. There will be a new GW-GW connection. The call info will be exchanged between the GW SBC's. Due to a mismatch in the call info, the detail calls are getting dropped.

Steps to Replicate: 

Scenario 1: The GW_GW link should be up between the D-SBC and GSX.

Scenario 2. Make three SBC_GSX calls.

Scenario 3. After calls are stabilized, run the SBC switchover.

Expected Results: All the calls should be stable after a switchover and properly completed.

By switching back to slot ID to the 5th bit, all flavors of the SBC use the sameslot ID bit.

Workaround: N/A