SBC Core 08.02.06R000 Release Notes

PortFix SBX-114842: Microsoft TEAMS Shared Line/Hold issue.

Impact: The SBC fails to send ACK after mix of INVITE replaces and refer (Microsoft Teams Shared line/hold) feature.

Root Cause: After received 200OK from a refer INVITE, the SBC fails to release tone announcement.

Steps to Replicate: This is a complicated setup that required Microsoft TEAMS Shared line/hold feature.

The code is modified to address the issue.

Workaround: Disable tone as announcement.

PortFix SBX-113170: The call fails with the NRM error "could not find a card for IP call".

Impact: Some calls start failing after a switchover and continued to fail until a manual switchover was performed.

Root Cause: There is no definitive root cause.

The current theory is that a race condition on switchover caused an interior table to contain bad data, which resulted in internal messages being sent to the wrong process. As a result, this prevents certain calls from completing.

Steps to Replicate: The steps cannot be reproduced. 

The code is modified to address the issue.

Workaround: This issue can be resolved by a manual switchover.

The G711 codec is used incorrectly during a G729 UPDATE.

Impact: The SBC played the incorrect media towards Ingress EP after a call is established.

Root Cause: Issue 1: Port was not updated after an UPDATE during tone play.

After the SBC receives an UPDATE from Ingress EP during tone play, the SBC updates the codecs and port information. But after a 200 OK for UPDATE is sent out, the SBC overwrites the new information with old information.

Issue 2: The wrong codec was used after the call is established.

After an UPDATE is received during a tone play, the SBC tends to suppress sending UPDATE/re-INVITE towards ingress during cut-thru (after tone play) by re-using the last tone codec on ingress leg.

However, it did not preserve/re-use the transcoding resources which resulted in wrong media played towards Ingress EP

Steps to Replicate: 

1. Configure the SBC with Basic call configuration.
2. Attach TONE_GENERATION_CRITERIA to Tone and Announcement Profile.
3. Configure earlyMedia in Egress TG.

Procedure:

1. The SBC receives INVITE with 0,8,18 from Ingress EP.
2. The SBC receives 180 Ringing with 0 from Egress EP.
3. The SBC receives UPDATE from Ingress EP with 18 and with new media port.
4. The SBC receives 200 OK for INVITE without SDP from egress EP.

Expected Results:

1. The SBC will forward the INVITE to Egress EP.
2. Once the SBC receives 180 with SDP, it will forwarded to Ingress EP
   The SBC starts monitoring RTP from Egress. Once timeout happens, the SBC will start generating tone towards Ingress EP.
3. Once an UPDATE is received, the SBC will change the codec to 18 and port to new media port.
4. The SBC receives 200 OK for an INVITE.
   The SBC will use the same codec combination used during tone play.
   A call gets established successfully as G729-PCMU.

The code is modified for the following issues: 

Issue 1: The SBC now changes the port towards Ingress EP.

Issue 2: The correct media plays the tone play Ingress EP, based on whether the call needs Transcoding resources are not.

Workaround: None.

The SMM is not working when a From header contains an escape character.

Impact: The SMM may fail to parse a display name in a double quote string, where the inside string may have more than one escape characters next to each other of a header.

Root Cause: Logical error in parsing logic that cause parsing fail.

Steps to Replicate: 

1. Configure a rule to access double quote string display name of uri header, or any parameter.
2. Include an input display name.

The code is modified to address the issue. 

Workaround: There is no workaround.

PortFix SBX-112445: Conference calls fail when taking a recorded (SIPREC) path, but they work when SIPREC settings are not enabled.

Impact: The SBC cleanup call occured during hold retrieval for the transferred call. This issue observed only when the SIP Rec is enabled.

Root Cause: Invalid operation on media resource during call hold/unhold for the transferred call leads call to cleanup. This issue observed only when the SIP Rec is enabled.

Steps to Replicate: 

1. Enable SIP Rec.
2. Make a blind/attended call transfer and after successful transfer, perform call hold and unhold.

The code is modified to not perform any invalid operation on media resources during call hold/unhold for the transferred call.

Workaround: NA.

PortFix SBX-112309: An LSWU on SWe (VMWare/KVM) via Platform Manager fails with the additional reboot of standby.

Impact: An LSWU from V09.02.02R001 to any higher release 1:1 HA SWe (KVM/VMWare) through Platform manager fails because the standby instance undergoes an additional reboot during the upgrade procedure.

Root Cause: Index Marker file was missing on the standby instance prior to the LSWU procedure. This is because, when the application comes up with the standby role, the Index Marker is created only when there is a mismatch in the calculated and DB populated indices.

Steps to Replicate: To reproduce this issue, use the following procedure:

1. Create 1:1 HA SWe on KVM/VMWare.
2. After the HA application comes in sync, run clearDB on the standby.
3. Perform an LSWU on the 1:1 HA SWe.

The code is modified so an Index Marker file is made on the standby instance irrespective of processor index mismatch in the calculated and DB populated indices.

Workaround: The workaround for the upgrade from 9.2.2R1 to any higher release:

Before initiating the upgrade procedure (from PM or CLI), user needs to run the following command as a root user from the linux shell of the active and standby instance.

touch /opt/sonus/conf/swe/capacityEstimates/.indexMarker

PortFix SBX-111719: The DNS Process core dumps.

Impact: The core observed in timer function when DNS lookup timer timed out.

Root Cause: The DNS TCB ID as changed when NAPTR query moved from the DNS server 1 to Dns server 2. This is causing the TCB pointer to be NULL, when timer function trying fetch TCB ptr using TCB ID.

Steps to Replicate: dnslookupTimeoutTimer = 20, retransmissionCount 15, retransmissionTimer 500, noPort5060 enabled

1. Have three DNS’s are configured.
2. Once a call is made, the SBC sends NAPTR queries to each DNS server.
3. Once it reaches to the third DNS server, no SRV queries are generated and the DNS process core dumped.

The code is modified to pass DNS TCB pointer instead of the TCB ID as setTimer function parameter and adding proper NULL check for tcb pointer in timer function before accessing it.

Workaround: No workaround.

PortFix SBX-108126: Observed a SAM Process crash in another active node during SIPFE crash testing.

Impact: The SAM Process coredumps due to a buffer overflow.

Root Cause: Key:value length of incoming Message is max 255 Bytes.
But in the yang spec, the buffer length was on 23 bytes. As a result of this issue, the display was truncated and also crashing due to buffer overflow.

Steps to Replicate: Steps:

1. Configure the testbed for the fault avalanche testing.
2. Trigger crash by sending fac-sipfe-0 in From header. From(calling Party) should have value > 23 bytes.

Expectation:

1. The current active should crash the SAM Process.
2. Standby will take over as active and blocks the SIP message that triggered crash.

The code is modified to handle 255 bytes string, so the buffer overflow does not happen and display is not truncated.

Workaround: NA

PortFix SBX-107976: Disable the FAC feature in M-SBC, SLB.

Impact: Non-SIP personalities should not have FAC feature. So when the Non-SIP instance coredumped, then FAC handler Process was invoked to generate a core file.

Root Cause: Do not have check of personalities of instances when setting core pattern.

Steps to Replicate: Verify the user defined core pattern is not set in /proc/sys/kernel/core_pattern when instances are spawned with the personalities M-SBC, SLB, MRFP irrespective of facState.
% set system faultAvalancheControl facState <enabled/disabled>

User Defined Pattern:

cat /proc/sys/kernel/core_pattern
|/opt/sonus/sbx/bin/CoreHandler -f /var/log/sonus/sbx/coredump/core.1.%e_%p.%t -t %i -p %p

Default Core Pattern:
cat /proc/sys/kernel/core_pattern
/var/log/sonus/sbx/coredump/core.1.%e_%p.%t

The code is modified to not to set the user defined core pattern for personality type MSBC, MRFP, and SLB.

Workaround: Disable the FAC Feature.
% set system faultAvalancheControl facState disabled

PortFix SBX-109616: A core dump was observed in the SCM process in 2vcpu system.

Impact: Fault Avalanche Control is enabled by default from release 9.2. If it is disabled, it can result in a coredump.

Root Cause: When the Fault Avalanche Control is disabled, the mem map file is disconnected, but the application can still try to access the mem map as the mem map pointer was not set to NULL.

Steps to Replicate: Disable the Fault Avalanche Control and run calls.

The code is modified so the mem map pointer is set to NULL to when feature is disabled to address the issue.

Workaround: Without this fix, avoid disabling the Fault avalanche Control.

PortFix SBX-108219: A SAM Process coredump was observed in SLB for 1000 cps Peering Call load.

Impact: In a load run, there was a core being observed in the SLBDEBUG call when it hits the default states as part of the SBC message processing.

Root Cause: In the SLB debug call, there was new line (\n) being passed as part of a string.

Steps to Replicate: Observed this issue when running an INVITE call load with PRACK enabled

The code is modified so the new line (\n) is not part of the debug string.

Workaround: There is no workaround identified for this issue.

PortFix SBX-109597: Observed an SCM process crash on the newly active SSBC during SWO testing.

Impact: While the SBC is running in sensitive mode, observed the SCM Process core during a switchover testing under the 1000 cps call load.

Root Cause: As part of switchover, all the connected calls are rebuilt on a newly active SBC. But, there may be a chance that the SBC can hold few stale entries related to active calls in the system, which will be cleaned up after some time. If any of these stale entries receive an unexpected events, that leads to a core dump.

Steps to Replicate: 

1. Run 1000 cps load with 120 CHT.
2. Perform a switchover by killing the PRS Process.

The code is modified to identify these events for the stale entries and ignore them to avoid a coredump when the SBC running in the sensitive mode.

Workaround: None.

Portfix SBX-109464: The leadership algorithm workaround for old Openclovis issue can causes a core dump.

Impact: The safplus_gms process crashes when coming out of a split brain.

Root Cause: Incorrect/inconsistent data results in the code asserting.

Steps to Replicate: The problem is not easily reproducible and is caused by HA link instability/flapping.

The code is modified so that the data is consistent.

Workaround: Fix the HA link stability issues.

PortFix SBX-115074: There were multiple SCM Process core dumps observed on the Agent SBC-Y .

Impact: A SCM core dump may occur when using the SIPREC.

Root Cause: The root cause is memory corruption caused by a double MemFree() in the SIPREC code.

Steps to Replicate: This bug was found by code inspection.

The code is modified to prevent a double MemFree().

Workaround: None.

Portfix SBX-100881: The SBC sends a release to the H323 side.

Impact: The SBC is sending a call release to the H323 side.

Root Cause: During the codec selection on H323 side, the SBC ran into some offer-answer collision due to ACK SDP from SIP side. As a result, the SBC is terminating the call.

Steps to Replicate: 

1. Make a call from SIP to H323 and the call gets connected.
2. Send a late media Re-INV from SIP EP to the SBC.
3. The SBC sends 200 OK with SDP to SIP EP.
4. SIP EP sends ACK with SDP to the SBC.
5. The SBC sends a FACILITY to H323 EP.
6. The SBC terminates the call.

The code is modified so that the codec selection on the H323 side occurs independently during a modify offer-answer is in progress.

Workaround: None

PortFix SBX-112322: The SBC cores when receiving a subscribe crankback failure.

Impact: When a relay subscribe tries to use crankback for the second route and if the SBC does not find trunk group, the SBC may core.

Root Cause: The SBC cores due to a duplicated memory freed.

Steps to Replicate: 

1. Configure two routes for relay Subscribe. The second route is invalid.
2. After the first route exhausted, the SBC tries to crank back for the second route. The SBC fails to find trunk group for a second route.

The code is modified to prevent a duplicated free.

Workaround: Correct the second route.

PortFix SBX-109324: Crankback does not work if egress TG is locally OOS.

Impact: This issue occurred only for domain based routing. In case a TG is locally set to out of service, the policy request sent to PSX was not proper for the next route. The domain name in called URI section of policy request was not updated. So the SBC was sending the old domain name in Policy request. As a result, the PSX server was sending previous route, not the current route.

Root Cause: In case a TG is set to out of service, the domain name in called URI section of policy request was not updated while sending policy request for next route.

Steps to Replicate: 

1. Configure multiple routes using domain based routing.
2. Set one of the routes to out of service.
3. Set force re-query flag to true in egress IPSP profile.
4. Make a call and send 3xx with multiple routes in contact header.
5. After this fix, the SBC should not stop at the out of Service TG. Instead it should try all contacts with TG state as In service.

The code is modified so the domain name in called URI section of policy request is updated properly.

Workaround: None.

PortFix SBX-112381: The SBC Crash-Application is in a rebooting loop.

Impact: A PRS core is occurring when the code is processing an ICE STUN packet and there are more than 20 Teams clients ringing. This could potentially happen in a simultaneous call group pickup scenario e.g. more than 20 users with a single device or 10 users with 2 or more devices - all ringing at the same time.

Root Cause: The root cause of the core is a bug in the code that handles the incoming STUN packet. This code is overwriting the end of array by writing too many entries into the array. This results in memory corruption and eventually a core.

Steps to Replicate: Run a call group pickup scenario with more than 20 users all having their Teams clients ringing at the same time.

The code is modified to prevent it from writing too many entries into the array.

Workaround: Disable media-bypass or reduce the number of users in the call group.

PortFix SBX-111743: The SBC blacklist IP was running with the wrong port.

Impact: ARS blacklists port 0, when an INVITE contains a Route header without a port number, and the 503 response contains Retry-After header.

Root Cause: The code is not supported when no port number is provided.

Steps to Replicate: 

1. Configure the ARS Retry-After profile.
2. Enable the callRouting useRouteSet received in ingress and egress trunk groups.
3. UAC sends INVITE containing Route without port number:
   Route: <sip:xxx.xx.xxx.xx;lr;dtg=SBXSUS1_LABSIP2>
4. UAS responds with 503 Service Unavailable with Retry-After header.
5. Use the CLI to check egress zone sipArsStatus.

The code is modified so when there is no port number is set, we now retrieve the port number from the transaction control block.

Workaround: Define an SMM rule that adds the port number (5060) if the Route header in the initial INVITE does not contain a port number.

PortFix SBX-112349: There was a SBC 132 Release code (MODULE FAILURE).

Impact: An Async CMD Error was reported by XRM triggered call being torn down with the release code 132 when running an audit call.

Root Cause: With the high level error messages, we can only determine that there is some kind of race conditions for certain call flow(s) when a RID modify command was received for a disabled RID. (RID = receive ID in NP).

Without call informatiuon, the root cause could not be determined.

Steps to Replicate: The steps cannot be reproduced.

The code is modified to address the issue.

Workaround: None.

PortFix SBX-111688: The ReqURI and TO fields in SIP INVITE were rolled.

Impact: When the Diversion header is presented in ingress INVITE without an RN in Request-URI, the egress RURI would use RN in username field, after PSX LNP dip.

Root Cause: A fix for a previous fix has caused a problem for the RURI username settings and resulted in this issue.

Steps to Replicate: 

1. Set up the SBC with an external PSX to make LNP calls.
2. Observe the  disableRn flag in egress IPSP in PSX and flag UseGAPwhenRnisDisabled in the SBC's egress TG. Test different combinations of < disableRn, UseGAPwhenRnisDisabled >
3. The Ingress INVITE needs to contain Diversion header without an  RN in Request-URI.

The code is modified to ensure the RURI always contains the correct username.

Note: When the egress IPSP has "SIP TO Header Mapping" set to "Called Number", the TO header's username is the ingress TO URI. The number is not be globalized. If the customer would like it to be globalized, the globalization profile of the egress TG could be modified to globalize TO URI. Another way is using SMM to modify it.

Workaround: Use the resolution in Warning-21-00029918.

PortFix SBX-105657: A SBC Upgrade failed.

Impact: The LSWU upgrade failed from 7.2.2R0 to 8.2.3R0

Root Cause: The /tmp partition was running out of disk space and restoration failed during the upgrade.

Steps to Replicate: Run an LSWU upgrade from 7.2.2R0 to 10.0.0.

Ensure the enough space available to address the issue.

Workaround: Free some space in /tmp dir and re-ran the upgrade.

PortFix SBX-110303: The Mgmt Port Status was showing the wrong Status.

Impact: An unconfigured Management port is shown as UP when there is no cable connected to the port.

Root Cause: The SBC did not keep track of management port status if the management port is not configured/used.

Steps to Replicate: 

1. Show the issue.
   1. Leave SBC5400 mgt2 port unconnected.
   2. Display mgmtPortStatus - the unconnected mgt2 shown as admnEnabledPortUp. This is the issue.
      admin@YF01> show table system ethernetPort mgmtPortStatus
      CE PORT IF NEGOTIATED DUPLEX RX ACTUAL TX ACTUAL RX TX RX
      NAME NAME INDEX MAC ADDRESS SPEED LINK STATE MODE BANDWIDTH BANDWIDTH PACKETS PACKETS ER
      -------------------------------------------------------------------------------------------------------------------------
      YF01 mgt0 1 0:10:6b:3:c8:d speed1000Mbps admnEnabledPortUp full 232 0 45 3 0
      YF01 mgt1 2 0:10:6b:3:c8:e speed1000Mbps admnEnabledPortUp full 0 0 6 2 0
      YF01 mgt2 3 0:10:6b:3:c8:f unknown admnEnabledPortUp unknown 0 0 0 0 0
      YF01 mgt3 4 0:10:6b:3:c8:10 speed1000Mbps admnEnabledPortUp full 0 142 2 42 0
      [ok][2021-05-18 10:32:15]
2. Show the fix.
   1. Leave SBC5400 mgt2 port unconnected.
   2. Display mgmtPortStatus - the unconnected mgt2 shown as admnEnabledPortDown (this is the fix).
      admin@YF01> show table system ethernetPort mgmtPortStatus
      CE PORT IF NEGOTIATED DUPLEX RX ACTUAL TX ACTUAL RX TX
      NAME NAME INDEX MAC ADDRESS SPEED LINK STATE MODE BANDWIDTH BANDWIDTH PACKETS PACKETS
      -------------------------------------------------------------------------------------------------------------------------
      YF01 mgt0 1 0:10:6b:3:c8:d speed1000Mbps admnEnabledPortUp full 248 0 271 6
      YF01 mgt1 2 0:10:6b:3:c8:e speed1000Mbps admnEnabledPortUp full 0 286 27 103
      YF01 mgt2 3 0:10:6b:3:c8:f unknown admnEnabledPortDown unknown 0 0 0 0
      YF01 mgt3 4 0:10:6b:3:c8:10 speed1000Mbps admnEnabledPortUp full 0 0 0 8
      [ok][2021-05-18 13:32:27]
      admin@YF01>
   3. Display alarms and make sure there is no alarm on mgt2 (un-configured management port)
      admin@YF01> show table alarms currentStatus
      ALARM CLEAR
      ID TYPE TIMESTAMP INITIAL TIMESTAMP COUNT DESC
      -------------------------------------------------------------------------------------------------------------------------
      1237 AUTOMATIC 2021-05-18T17:27:53-00:00 2021-05-18T17:27:53-00:00 1 System Policer Alarm Level: Minor, Policer
      [ok][2021-05-18 13:32:30]
      admin@YF01>

The code is modified to keep track of the management port state changes even if there is no Management IP interfaces on the management ports.

Do not generate management port down event if there is no Management IP interfaces on the management port.

Workaround: None. Optionally, connect unused management ports to Ethernet switch to reflect the port UP status.

Portfix SBX-108325: The SBC failed to split the SBC2, and the SBC2 did not take over

Impact: The safplus_gms process crashes when coming out of a split brain.

Root Cause: Incorrect/inconsistent data results in the code asserting.

Steps to Replicate: The steps cannot be reproduced. 

The code is modified so that the data is consistent.

Workaround: Fix HA link stability issues.

PortFix SBX-113471: Multiple SBC core dumps were detected.

Impact: The SBC was forking hashtable corrupted memory.

Root Cause: Possibility of forking control fails to remove from hashtable when there is a resource cleanup.

Steps to Replicate: The steps cannot be reproduced.

The code is modified to ensure the forking call is removed from hashtable when free.

Workaround: None. 

PortFix SBX-110247: The "sonusSbxNodePolicerMinorAlarmNotification" alarm is generated by the SBC.

Impact: Packets arriving on a unallocated port do not get marked as rogue media.

Root Cause: The issue occurs only on ports that were used and disabled. Packets arriving on this port get marked as grace-media packets. This is due to a very high value of grace period is being incorrectly programmed for the udp port instead of the default 4 seconds. The high value is because of endianness.

Steps to Replicate: 

1. Reboot the VM.
2. Make a single normal call and wait for the call to end. Note down the RTP UDP port of the SBC used in the call.
3. After the call has ended, stream UDP packets to the same RTP UDP port previously used in the SBC call.
   We do not see any unalloctedport rogue media entry for the stream.
4. Stream UDP packets to a UDP port that has not been previously used in the SBC call.
   We will see rogue media entry for the stream.

The code is modified to perform the appropriate endianness translation on the affected fields to arrive at the correct value.

Workaround: No workaround. This does not affect normal media processing, or associated statistics.

PortFix SBX-110003: The SBC5400 coredump after MSRP call with direct media

Impact: A PRS core was encountered when customer was running MSRP with direct media and MSRP Multiplexing was enabled.

Root Cause: The root cause is when the the MRM code is using an invalid index to get a pointer to an array element.

Steps to Replicate: This is not reproducible as it is most likely triggered by a race condition.

The code is modified the ensure that MRM uses the correct index when attempting to get a pointer to an array element.

Workaround: The only known workaround is to avoid running MSRP with direct media and MSRP Multiplexing.

Portfix SBX-109336: The SBC 5110: BIOS not updated to 2.07 post-upgrade as indicated in Release Notes.

Impact: The BIOS is not upgrading part of the SBC application upgrade.

Root Cause: The flashroom utility is using /dev and /proc file system to upgrade BIOS. these file systems are un-mounted part of grub2 config upgrade.

Steps to Replicate: Upgrade the SBC app from 7.x (BIOS=2.6) to >= 8.1.x.(BIOS = 2.7). The BIOS should upgrade part of an application upgrade.

Mount /dev and /proc file system before calling BIOS upgrade

Workaround: 

1. Stop SBC App
   1. #sbxstop
2. Go to /opt/sonus/bios-update/
   1. #cd /opt/sonus/bios-update/
3. Run the script to update BIOS.
   1. # ./updateBIOS.sh bios5X00_v2.7.0-R0.rom

This scripts will take few minutes to update BIOS, after its done reboot host. In next boot it will boot up with new BIOS.

PortFix SBX-108388: The SBC set "user=phone" on Dummy History-Info header.

Impact: When interworking between diversion header and history info header if the diversion count is more than 2 then the SBC generates dummy history info headers. These dummy history info headers contain user=phone which some end points reject.

Example of a dummy history info header.

History-Info: sip:unknown@unknown.invalid;cause=302;user=phone;index=1.1;mp=1

Root Cause: The code was mistakenly adding the user=phone attribute even when no phone number was present in the dummy history info header.

Steps to Replicate: Configure the SBC to support interworking from diversion header with a count of 5 to history info headers and check that the dummy history info headers do not include user=phone.

The code is modified not to include user=phone in the dummy history info headers.

Workaround: Use SMM to remove the user=phone attribute from the dummy history info headers.

PortFix SBX-108225: Observed core files on fresh installed VMware SWe.

Impact: The SSREG and SLWRESD processes a coredump on VMware SWe due to the time being set in past.

Root Cause: When the SBC is installed using ISO and then app is installed, the ntp sync is occurring while the application is running and in case the time is set in the past, it causes SSREQ and SLWRESD processes to core dump.

Steps to Replicate: Launch with the NTP IP other than a default IP, check if a coredump occurs due to time being set in the past.

The code is modified to update the /etc/ntp.conf before the SBC comes up so the runntpdate can access the NTP server IP.

Workaround: No workaround.

PortFix SBX-108237: Performance: Observed SAM and SCM process coredump for FAC enabled execution on SBC 5400

Impact: The SCM experiences a core dump when the Invite gets a transaction timeout and the 200 Ok for Invite is received at the same time.

Root Cause: In the problem scenario, the SBC is trying to send ACK for the 200 OK, the SCM cores when creating a SIP Transaction for ACK Message.

Steps to Replicate: 

1. Run a Basic Invite call flow.
2. Do not answer for Invite on the UAS side.
3. After 32 seconds send 200 OK for Invite from the UAC side.

A race condition might occur and result in coredump.

The code is modified not to send ACK in this scenario as the CseqType and CseNumber are unknown and 0 respectively.

Workaround: Not Applicable.

PortFix SBX-96247: An SCM Process core occurs in the standby SBC after switchover while testing SIPREC with ARS.

Impact: A newly-active system may core upon the transition to the active SBC in either of these two circumstances:

1) ARS profile is configured on the active, but not the standby SBC.
2) PATHCHECK is in use when ARS is not configured

This core will rarely happen - even under these circumstances - because an internal race condition is also necessary for it to occur.

Root Cause: The code that handles blacklisted endpoints when a system is transitioning to active frees a chunk of memory and then continues to attempt to access this memory.
If the memory is re-allocated and re-used in the between the MemFree and the next reference to this memory, we will encounter unexpected results.

Steps to Replicate: A non-reproducible race condition is required in order to replicate this condition.

The code is modified so that the SBC no longer accesses the memory after it has been freed.

Workaround: N/A

Portfix SBX-108612: An SCM Process core dump occurred when INVITE with message size of 31700 bytes is sent

Impact: Receipt of SIP messages with Content-Type: multipart/mixed and more than 10 content entries cause coredump, if a message manipulation rule is active with criterion on message body.

Root Cause: Missing handling for unexpected message received.

Steps to Replicate: 

1. Configure and apply a message manipulation rule with criterion messageBody.
2. Send in INVITE with Content-Type: multipart/mixed and more than 10 content entries.

The code is modified so that if more than 10 content entries are present, the message is not considered for SMM rule actions with criterion on message body.

Workaround: None.

Portfix SBX-109922: The buffer used while printing PSP's was running out. As a result, this is an error.

Impact: The buffer used while printing PSP's was running out because of large PSP. As a result, this is an error.

Root Cause: The buffer used while printing PSP's was running out. As a result, this is an error.

Steps to Replicate: 

1. Run the suite with the RTCP Interworking Enhancements.
2. The warning above in the Opensaf logs should not come.

The code is modified to accommodate bigger strings.

Workaround: None.

Portfix SBX-107133: Max FD limit is reached in SLB beyond 7,04,000 TLS connections (Access Registrations) tried.

Impact: Max FD limit is reached in the SLB beyond 7,04,000 TLS connections (Access Registrations) tried.

Root Cause: Observing the FD congestion at high load due to FD limits reached.

Steps to Replicate:

1. SLB Flavor = 32vCPU_64GBRAM_100GBHDD
2. Run REGISTER PERFORMANCE with more than 5,12,000 REGISTERS (TLS) and EGRESS (UDP) 
3. SLB is restarting after >5,12,000 REGISTER TLS CONNECTIONS

The code is modified for the max FD for the KVM and SLB.

Workaround: None.

PortFix SBX-108270: The S-SBC sends a DNS SRV before completing all NAPTR query.

Impact: The S-SBC sends DNS SRV before completing all NAPTR query.

Root Cause: A DNS agent request timeout occurs after 10 seconds. Because the DNS agent sends the DNS lookup (NAPTR) request to DNS Client process, the DNS client process then queries to an external DNS server for NAPTR records. The DNS agent waits until 10 seconds then timeout's for each DNS lookup. As a result, the DNS request is failed with the first DNS server and query is still in process for a second DNS server.

Steps to Replicate: Configure 2 DNS server's

1. Configure DNS global configuration to:
   admin@SBX136 % set global servers DNS global
   retransmissionCount 14;
   retransmissionTimer 500;
   set addressContext default dnsGroup DNSGrp1 negativeDnsCacheSupport disabled
2. Configure dnslookupTimeoutTimer to 10 sec.
3. Stop the DNS server.
4. Make a call.

Observation:

1. DNS should retransmit DNS NAPTR query to DNS server1 and after should move to DNS server2.
2. After 10 seconds, NAPTR query will stop.
3. The SBC should send SRV/A query to DNS Server2.

The code is modified to:

1. Add new configuration timer flag "dnslookupTimeoutTimer" for DNS lookup.
2. Once the lookup timer expires, the DNS client will stop sending same query to DNS server (Ex: if NAPTR query is re-transmitted towards the DNS server, after timer expire. NAPTR query will be stopped).

Workaround: None.

PortFix SBX-111100: Host check validation failing - Required GB RAM 6 but found 0.03125.

Impact: The few host machines in the AWS data center use different units for RAM, HostCheck script assumes that available memory is given in MB. The HostCheck script is fixed to calculate available memory based on unit of memory.

Root Cause: The HostCheck script does not have code to consider memory unit.

Steps to Replicate: Create VM with >=32 GB RAM, application should come up without complaining about memory in VM.

The code is modified to calculate available resources based on units.

Workaround: Create instance/VM that has < 32 GB RAM.

PortFix SBX-105370: The Active Register per TG shows more than the total stable registration.

Impact: Under some condition(s), the ingress zone’s activeRegs count can be incremented and not decremented when the registration terminates.

The causes the ingress zone’s activeRegs count to grow incorrectly, and never reach ZERO (even after all registrations are terminated).

Root Cause: The SIPFE and SIPSG RCB allocation/deallocation become out of sync, indirectly causing the ingress zone's activeRegs count to increment and not decrement.

Steps to Replicate: Perform REGISTER/401 - REGISTER/403 until the ingress zone activeRegs count issue is detected.

The code is modified to correctly handle the SIPFE's registration bind timer expiration. 

Workaround: None.

PortFix SBX-110350: The SBC is forming invalid packet where it is adding 00 in around all headers and parameters.

Impact: The SBC puts a NULL termination in every parameter and header of the message. In this case, there was a parser error on a parameter when the DNS translation was required.

Root Cause: During the parsing of the message after a DNS query, if a parser error occurs, then incorrect termination is put/left in the message when sent on the wire.

Steps to Replicate: 

1. An incoming INVITE had a known syntax error related to alert_info, configuration was to parse through the filterProfile and not transparently forward alert_info. This resulted in parse error and bad formatting when DNS was executed.
2. Configure the alertInfo to transparently pass on egress, and DNS function doing fqd/IP swap executes correctly and egress message is sent with good formatting.

The code is modified to:

1. Log parse an error line number and pdu message.
2. Apply the filterProfile when parsing the message. The customer needed to add filterProfile to transparently forward the parameter failing parsing on egress to avoid parse error and avoid incorrect parameter termination.

Workaround: On the Ingress, apply an SMM to rename alert_info to an unknown (x-alert_info), and rename back on the egress.

PortFix SBX-109244: The SBC Configuration Manager shows "no records found" for SIP TG.

Impact: The SBC Configuration Manager shows "no records found" for SIP TG.

Root Cause: There is no OAM check in the EMA code.

Steps to Replicate: 

1. Log onto EMA.
2. Navigate to Sip Trunk group.

As a result, we can see the total number of records.

The code is modified to address the issue. 

Workaround: None.

PortFix SBX-112307: A customer's SBC 7000 SIPREC metadata did not contain an ‘AOR’ parameter value.

Impact: In the SIPREC, the metadata sender and receiver's AOR fields for the tel URI were not populated properly.

Root Cause: Due to defect in the design, these fields were populated incorrectly.

Steps to Replicate: 

1. Configure the SBC with SIP Rec server.
2. Run a basic call with "tel" URI in From and To header.

The code is modified to populate as per requirements/standard.

Workaround: Not Applicable.

Portfix SBX-110354: After upgrading from V08.02 or V09.02 to 10.00.00R000 successfully, a REVERT from 10.00.00R000 to V08.02.04R000 is failing.

Impact: Reverting from the higher software version to lower software version fails.

Root Cause: The OAM does not upload restored revision against lower software version (ie. restore of config tar ball that was created on lower software version before upgrade). After following the MOP for downgrade when OAM comes up, it downloads the last revision uploaded on EMS (in this case config revision for higher software version). As a result, it fails to start the service because of a version mismatch.

Steps to Replicate: 

1. Spawn OAM + S-SBC on V08.02.04R000.
2. Create multiple revision (e.g revision 1, 2, 3, 4).
3. Upgrade to software version V10.00.00R000.
   The revision 5 gets created from revision 4,
   Create revision 6, 7, 8.
4. Restore to revision 4 -> revision 9 will get created and uploaded to EMS.
5. Downgrade to revision V08.02.04R000.

Expected O/P:

The OAM should come up with revision 10 created from revision 9.

The code is modified to upload a configuration when a revision that is created on a lower software version is restored. And after a  downgrade, it picks the revision with software version where the downgrade is done.

Workaround: None.

PortFix SBX-107753: After the LSWU, the apache2 not coming up, resulting in an EMA or PM access issue.

Impact: After an upgrade, the Apache did not start.

Root Cause: Apache did not start due to a timeout while requesting a password. The Apache would need to be restarted manually especially in case of a failed upgrade.

Steps to Replicate: 

1. Perform an upgrade (LSWU/standalone).
2. If upgrade has failed, verify that the apache has come up properly with default keys and cert.
3. If upgrade has succeeded, verify that first Apache came up properly with default keys, then after app start, the installed keys and certs are being used by the Apache.

The code is modified to find any occurrence of the one-time-issue of server key getting corrupted in the future.

Workaround: In case of a failed upgrade or apache startup failure restart apache or apply the workaround mentioned in an issue relating to the apache2 failing after an LSWU. 

PortFix SBX-106316: The call established prior to a switchover fails when put on hold after a switchover.

Impact: The call established prior to a switchover with a dynamic payload type fails when put on hold after switchover.

Root Cause: After a switchover, while processing the hold request, it was unable to locate the codec due to an issue in reconstruction of some flags in PSP data.

Steps to Replicate: 

1. Have a HA setup.
2. In IPSP, enable "Minimize relay of media changes from other call leg" flag and "lockdown preferred codec" is enabled.
3. Establish a call with dynamic codec like AMR.
4. Perform a switchover, once standby becomes active send a call hold request.
5. After a short period of time, send an call-un-hold request.

result: Call hold invite should be successful with 200 ok for that and call un-hold should also be successful.

The code is modified so that necessary flag for dynamic payloads in PSP data are reconstructed properly.

Workaround: Disable“Lock Down Preferred Codec” and enable “Relay Data Path Mode Changes".

PortFix SBX-107747: During Cyclic switchover tests, a PRS Process coredump was observed on the MSBC1.

Impact: The application on both active and standby switched over and application reset in short order of each other. Core files were created on both sides of the HA pair.

Root Cause: The issue occurs during multiparty call processing, when the SBC tries to determine whether a message was destined for an ingress or egress call segment. As a result of running multiparty call processing, the code was accessing the pointer to the multi party call resource after it was freed up and set to NULL.

Steps to Replicate: This issue is not reproducible. Potentially due to a race condition in the code.

Check the multi party call pointer is not NULL before reading from it to avoid the coredump.

Workaround: There is no known workaround for this issue.

PortFix SBX-112561: The regexp string "\r\n" was not exported by “user-config-export”.

Impact: The \r\n in regex is lost while exporting a configuration using the user-config-export CLI command.

Root Cause: XML formatting trims empty node values.

Steps to Replicate:

1. Create an SMM Profile using the following commands:
   set profiles signaling sipAdaptorProfile ESMM state disabled
   set profiles signaling sipAdaptorProfile ESMM advancedSMM disabled
   set profiles signaling sipAdaptorProfile ESMM profileType messageManipulation
   set profiles signaling sipAdaptorProfile ESMM rule 1 applyMatchHeader one
   set profiles signaling sipAdaptorProfile ESMM rule 1 criterion 1 type message
   set profiles signaling sipAdaptorProfile ESMM rule 1 criterion 1 message
   set profiles signaling sipAdaptorProfile ESMM rule 1 criterion 1 message messageTypes requestAll
   set profiles signaling sipAdaptorProfile ESMM rule 1 criterion 2 type header
   set profiles signaling sipAdaptorProfile ESMM rule 1 criterion 2 header
   set profiles signaling sipAdaptorProfile ESMM rule 1 criterion 2 header name WWW-Authenticate
   set profiles signaling sipAdaptorProfile ESMM rule 1 criterion 2 header condition exist
   set profiles signaling sipAdaptorProfile ESMM rule 1 criterion 2 header numberOfInstances number 1
   set profiles signaling sipAdaptorProfile ESMM rule 1 criterion 2 header numberOfInstances qualifier equal
   set profiles signaling sipAdaptorProfile ESMM rule 1 criterion 3 type parameter
   set profiles signaling sipAdaptorProfile ESMM rule 1 criterion 3 parameter
   set profiles signaling sipAdaptorProfile ESMM rule 1 criterion 3 parameter condition exist
   set profiles signaling sipAdaptorProfile ESMM rule 1 criterion 3 parameter paramType generic
   set profiles signaling sipAdaptorProfile ESMM rule 1 criterion 3 parameter name realm
   set profiles signaling sipAdaptorProfile ESMM rule 1 action 1 type header
   set profiles signaling sipAdaptorProfile ESMM rule 1 action 1 operation regsub
   set profiles signaling sipAdaptorProfile ESMM rule 1 action 1 headerInfo headerValue
   set profiles signaling sipAdaptorProfile ESMM rule 1 action 1 from
   set profiles signaling sipAdaptorProfile ESMM rule 1 action 1 from type value
   set profiles signaling sipAdaptorProfile ESMM rule 1 action 1 from value 172.xx.xx.xxx
   set profiles signaling sipAdaptorProfile ESMM rule 1 action 1 to
   set profiles signaling sipAdaptorProfile ESMM rule 1 action 1 to type header
   set profiles signaling sipAdaptorProfile ESMM rule 1 action 1 to value WWW-Authenticate
   set profiles signaling sipAdaptorProfile ESMM rule 1 action 1 regexp
   set profiles signaling sipAdaptorProfile ESMM rule 1 action 1 regexp string "\r\n"
   set profiles signaling sipAdaptorProfile ESMM rule 1 action 1 regexp matchInstance all
   commit
2. Run the CLI command to the export configuration:
   user-config-export test.xml /profiles/signaling/sipAdaptorProfile
3. Delete the exported profile from CLI with the command:
   delete profiles signaling sipAdaptorProfile ESMM
   commit
   user-config-import test.xml
4. Run the following command:
   show configuration profiles signaling sipAdaptorProfile | display set | match string
5. Check if the regex string field, restores the original value.

Use the external XML tool called xmllint. 

Workaround: Manually edit the field in xml file after export.

Portfix SBX-108127: 9.1r3 to 10.0 LSWU failed with the reason "Failed_to_install_or_configure_database"

Impact: The LSWU failed with the reason "Failed_to_install_or_configure_database".

Root Cause: Ownership for the files in /var/log/postgresql is getting changed at the time of the upgrade.

Steps to Replicate: Perform an upgrade from any version to 9.x

The code is modified to restore ownership of the log files in /var/log/postgresql to postgress.

Workaround: None.

PortFix SBX-106329: The SBC disconnects a call every few seconds after recovering a DNS server.

Impact: The SBC disconnects a call every few seconds after recovering a DNS server.

Root Cause: The DNS client sends a failed lookup response to a DNS agent ( SCM ID 03,01 ...) when a probe query fails to get a response for a blacklisted DNS server because a probe query and regular query were used the same FQDN, record type and zone id.

The timeout response for probe query in the DNS client process as triggered DNS failed response towards SCM process.

Steps to Replicate: DNS settings: Three DNS servers are configured. Each weight is set to 50.

DNS#1 10.xxx.x.xxx has RRs with ttl=5
DNS#2 10.xxx.x.xxx has RRs with ttl=0
DNS#3 10.xxx.x.xxx has RRs with ttl=0

<Time series>
DNS#1 10.xxx.x.xxx process was down (Pkt No.29615). DNS#2 and DNS#3 were available.
After few Seconds DNS#2 10.xxx.x.xxx process was down. Only DNS#3 was available.

The code is modified so the DNS probe query does not trigger any response towards a DNS agent from DNS client.

So do not send any failure response to the DNS agent in case of probe query failure.

Workaround: None.

PortFix SBX-110833: There are call trace logging issues while all Call Trace filters are disabled.

Impact: Messages traced at level 4 for sageTracing may erroneously have FILTER=<name> in the trace header. For the sageTracing, the filter name should be blank.

With the sageTracing enabled, 70% of the received INVITES are traced at the level 4 and 0.5% of calls have all their messages traced at level 4. The sageTracing is tracing collected for the Strategic Analysis Gap Execution.

Root Cause: The filter name messages traced for sageTracing should be blank but instead uses the 64th entry in the filter names table.

Steps to Replicate: 

1. The sageTracing is enabled.
2. Create at least 64 call trace filters and enable them.
3. Delete the trace filters.
4. Send INVITE messages.

The code is modified to filter the name blank.

Workaround: None.

Portfix SBX-106475: The Cloud SBC instances are not coming up after a fresh installation or rebuild and the MGMT I Pis unreachable in BLR-PC3.

Impact: The SBC instances are not coming up after a fresh installation or rebuild and the mgmt ip is unreachable when using the X710 sriov vfs for pkt interfaces.

Root Cause: The X710 driver reset has been known to experience a delay in completing, thus causing a cloud-init service failure. For example, after the OS boot (as part of renaming the interfaces by sonusdev), the X710 driver is restarted. When the X710 does not immediately reset, the next cloud-init service fails, and the system is not in proper state.

Steps to Replicate: The SBC instances should come up properly after fresh installation or rebuild and mgmt ip should be reachable with all supported sriov vfs for pkt interfaces.

The code is modified so after the renaming is done and the network driver is restarted, wait for some seconds to check if all the nic ports are up and running in sonusdev itself, before exiting the service. This blocks all other dependent service from starting. Once the nic port are up, proceed with the system bring up. With these code changes, the nic does not come up then, we log it as critical log and expect the user to check on it.

Workaround: None.

PortFix SBX-107374: A discarded UPDATE of media addition when a data connect TCP stream calls.

Impact: When an UPDATE is received with new additional stream, the S-SBC sends an alloc request to a different M-SBC instead of sending to the M-SBC that processed the initial INVITE. This is results in resource allocation failure and S-SBC sends 488 response.

Root Cause:When an UPDATE is received with new additional stream, the S-SBC sends an alloc request to a different M-SBC instead of sending to the M-SBC that processed the initial INVITE. This is results in resource allocation failure and S-SBC sends 488 response.

Steps to Replicate: The steps cannot be reproduced. 

The code is modified to ensure that new allocation requests are sent to same M-SBC if there the prior allocation requests are processed by a particular M-SBC.

Workaround: No workaround

PortFix SBX-107517: Import configuration results in two different final status depending where you look at it.

Impact: The postgres DB restore fails that leads to provisioning issues such as:
The "show configuration" does not display a complete configuration related to policy entities like sipTrunkGroup, IpPeer etc.
The "delete" operation fails for sipTrunkGroup and other policy related entities. The exportConfig does not contain policy related entitles.

Root Cause: The PostgressDB restore is failing due to restricted permission.

Steps to Replicate: Set up required: 1to1 S-SBC virtual platform and register it within Direct-Single type of cluster on EMS.

1. Configure 2 TG and saveAndActivate revision (e.g. revision 2).
2. Configure couple of more CLIs and save AndActivate (e.g revision 3).
3. Restore revision 2 either from CLI or EMS.
4. After restore is successful. try to delete a TG. Observed Application failure.

After implementing the fix, use the 4 steps and as a result, the delete was successful.

The code is modified to permit an appropriate permission to postgres user while restoring the pg_policy dump.

Workaround: Use the following steps:
#cd /opt/sonus/sbx/scripts

Update the oam-config.sh on active and standby the SBC as following:

replace
$CHMOD -R 660 $extractDir/

with

$CHMOD -R 665 $extractDir/
accessRights=`$STAT -c %a $SONUS_TMP_DIR`
$CHMOD 775 $SONUS_TMP_DIR

find $RM -rf $extractDir and add following line below this

$CHMOD $accessRights $SONUS_TMP_DIR

Restore a revision that has pg_policy_xx.dump with correct Trunkgroup, ipPeer etc. entries.

Portfix SBX-110066: Observed a SAM Process core dump in SLB while testing Performance with SLB (TLS Peering Calls) - One time Occurrence.

Impact: A SAM Process core was observed on a shutdown.

Root Cause: A race condition in the process shutdown code occurred allowing access to an invalid pointer, and causing a core dump during the shutdown of the SAM Process.

Steps to Replicate: Restart the SBC and look for a SAM Process core.

The code is modified to avoid the race condition that led to the core.

Workaround: No workaround, but since this core is during shutdown, there is no impact to normal operations of the SBC.

PortFix SBX-106058: Multiple coredumps were observed during a Load run for generated Subscription call Flow tested during Registration Display Enhancement feature.

Impact: Multiple coredumps were observed when Reg-Event Subscription feature flag gets enabled.

Root Cause: Core 1: Incorrect type cast of a structure led to invalid memory access and core was observed.

Core 2: During a switchover, the list "pendingCcDsIcmList" was never initialized; however, it was incorrectly getting freed.

Core 3: Accessing of a NULL pointer led to this core.

Steps to Replicate: Load test for Reg-Event Subscription initiation.

Core 1: The code is modified to point to valid memory.

Core 2: In case of a switchover, the code is modified to initialize the list correctly.

Core 3:
Before accessing the pointer, the code is modified to resolve the issue.

Workaround: N/A

PortFix SBX-110248: The SBC SWe crashes when making a video call with a certain type of phone. 

Impact: The SBC SWe NP crashes when IPsec fragmented signaling packets scenario calls made by using IPsec crypto as NULL encryption.

Root Cause: During the IPsec crypto null processing of fragments (chained mbuf segments) by the SWe NP, an incorrect first segment length corrupted adjacent buffers, which resulted in a crash of the SBC SWe.

Steps to Replicate: Establish IPsec session with null encryption and make calls.

The code is modified to address the issue. 

Workaround: Use IPsec non-null encryption suites such AES/3DES.

PortFix SBX-107430: URI parameter setting order of R-URI does not meet RFC 3966 requirements.

Impact: In the request URI, the isub parameter was after the NPDI parameter, which does not meet the RFC 3966 requirement. (ie. npdi;isub=1111)

Root Cause: In the code (sipsgCallProcEngine.c), the isub parameter was getting populated after NPDI and other user parameter.

Steps to Replicate: 

1. Send an INVITE with isub and npdi in the R-URI
2. SBC forwards INVITE to egress
3. The isub and npdi parameters are interchanged in the egress R-URI

The code is modified so the isub parameter populating function is moved before all other parameter populating functions.

Workaround: NA

Portfix SBX-109327: Some CDR Events in ACT files have timestamps that are not current.

Impact: Duration Field (+24 Duration of SIP Registration/Subscription Context) is not displayed for some register transaction.

Root Cause: Duration Field (+24 Duration of SIP Registration/Subscription Context) is only updated in case of de-register for Successful transaction.

Steps to Replicate: 

1. Configure SBC for Registration/Subscription.
2. Perform following Registration.
   A ------REGISTRATION----->SBC-----REGISTRATION---->B
   A<--------404--------------------SBC<-----------404---------------B
   A ------REGISTRATION----->SBC-----REGISTRATION---->B
   A<--------200OK--------------------SBC<-----------200OK---------------B
3. Check CDR for "Duration Field" ( +24 Duration of SIP Registration/Subscription Context)

The code is modified to display the duration of SIP Registration/Subscription context for every CDR event.

Workaround: None.

PortFix SBX-112624: The SBC is unable to recognize a PRACK message, when call hunts to a secondary route.

Impact: The PRACK was rejected with a 481 when the end-to-end PRACK is active if a call is cranked-back following a successful end-to-end PRACK on the earlier route.

Root Cause: If end-to-end PRACK is performed on the first route and then a late crank-back occurs, subsequent PRACK is rejected because stale information is present from the first route.

Steps to Replicate: 

1. Ensure the end-to-end PRACK is supported.
2. Make a SIP-SIP call that routes and 18x is received on the first route. As a result, the PRACK procedure is successful.
3. Have the call fail on the first route, e.g. with 404, and the SBC configured to crank-back to a second route.

The 18x is received on the second route. When the calling party responds with PRACK, the call has a  481 in response.

The code is modified so that end-to-end PRACK information is started fresh for each route.

Workaround: None.

PortFix SBX-106760: Performance: While running IMS AKA Registrations on SWe KVM, cannot achieve 1000 subscribers with 30 RPS properly.

Impact: The SBC was not able to achieve the IMS AKA registrations even at a lower rate (30 registrations per second).

Root Cause: For one of the IPSec features, the XRM code was modified to perform route lookup and next hop destination MAC resolution during IPSec SA setup. This was causing a delay in setting up IPSec SAs during load causing timeouts during the IMS AKA registration.

Steps to Replicate: Run the IMS AKA Registration load at 30rps.

The code is modified to not invoke the route loopup and destination MAC resolution code that is causing delay in setting up IPSec SA, which addresses the issue.

Workaround: None.

Portfix SBX-110184: Performance: While pumping Emergency call load on top of normal call load, HPC calls are getting rejected with 488 sip Error due to dsp overload even though resources were reserved on Openstack D-SBC HA.

Impact: The HPC calls are rejected with 488 even though the SBC has sufficient DSP resources reserved for such calls through the highPriorityCompressionReservation configuration.

Root Cause: In a D-SBC setup, when DSP allocation request is received, the T-SBC applies the call admission policy to know whether the call can be admitted or not before proceeding with DSP allocation. In this process, the T-SBC was not considering if the allocation request was for HPC call or normal call. As a result, the HPC calls were treated like normal call leading to rejections if this T-SBC is running under high load.

Steps to Replicate: 

1. Reserve DSPs for HPC calls.
2. Run a normal call transcoding at 200cps.
3. Run HPC transcoded calls at 5cps.
4. Ensure the HPC calls work as long as reserved DSP resources are not exhausted.

The code is modified to give preferential treatment to HPC calls during call admission policy at the T-SBC.

Workaround: No workaround.

Portfix SBX-107182: Performance: Observed "154 02172021 101624.655260:1.01.00.07241.MAJOR .NRS: ARP lookup for 172.10.2.3 on interface pkt0.2 with addrContextId 1 failed: SIOGARP error , error 6" MAJOR Logs on SBC while running.

Impact: The MAJOR logs related to the ARP lookup was getting logged in the .DBG files, when the IMS AKA registration load is run.

Root Cause: The log is generated when there is no ARP entry in the ARP table. The debug log was generated for each UE IP, for which there is no ARP entry. For one of the IPSec features, the XRM code was modified to perform a route lookup and destination MAC resolution during the IPSec SA setup. The ARP lookup is done as part of MAC resolution.

Steps to Replicate: Run IMS AKA Registration load.

The code is modified to not invoke the route loopup and destination MAC resolution code that is causing delay in setting up IMS AKA registrations, which addresses the issue.

Workaround: None.

Memory leak since upgrading to 8.2.5R0.

Impact: High memory utilization.

Root Cause: Two leaks of the same structure exist:

1. A memory leak is seen when a relayed SUBSCRIBE message is cranked back and an Alternate Server cannot be found.
2. The code that handles relayed messages may cause a leak in certain error scenarios.

Warning-21-00029922 pertains to this issue.

Steps to Replicate: 

1. This memory leak that may get triggered if a relayed SUBSCRIBE is cranked back and then an Alternate Server is not found.
2. We were unable to reproduce this error scenario which causes a leak of the Relay Control Block.

1. The code is modified to take the correct path when an Alternate Server cannot be found while handling a relayed message that has been cranked back.
2. Code has been added to start a timer when we begin processing a relayed messsage. In error cases, the Relay Control Block will be free when the timer expires - preventing the structure from leaking.

Workaround: None.

Once the memory utilization reaches 91%, an automatic switchover occurs. To avoid an automatic switchover, Ribbon recommends performing a manual switchover during a low traffic period. 

The SBC switchover and isolation of node/SVWSBCD.

Impact: The PRS core was truncated because a second ABRT was sent to the process.

Root Cause: The PRS core was truncated because a second ABRT was sent to the process while it was in the process of writing the core.

Steps to Replicate: This issue is not reproducible.

The code is modified to prevent sending two ABRT signals to the same process

Workaround: There is no workaround.

Missing the CDR for Teams call transfer scenario.

Impact: For an MS Teams blind transfer scenario with tonesAsAnnouncements enabled, where the MS Teams user A establishes a call to PSTN user B, then MS Teams user A establishes a call to user C and then invokes a blind transfer to establish the call between B and C. When the call is subsequently cleared one of the CDR STOP records for the call is not generated.

Root Cause: When the tonesAsAnnouncements are enabled, after a blind transfer is initiated from user A, the subsequent call clearing logic is not correctly deactivating the internal announcement resources causing the call to not fully clear internally which results in STOP record not being generated.

Steps to Replicate: With the SBC configured for MS Teams having DLRBT with announcementBasedTones enabled.

Procedure
--------------

1. Establish a call from MS Teams User A to PSTN user B.
2. Establish a call from MS Teams user A to PSTN user C.
3. Initiate a blind transfer from MS Team user A, between user B and user C.
4. Clear the call and check CDR records.

Expected Results
-----------------------
Calls, transfers, and announcements should be successful.

The SBC should generate START and STOP CDR records for A-B and A-C call.

The code is modified to correctly deallocate announcement resources for this call scenario so subsequent call clear can complete successfully and generate the correct CDR stop records.

Workaround: If tonesAsAnnouncements are disabled the call scenario works as expected.

A SWe media Issue occurred after an active reboot

Impact: Media issue after switchover.

Root Cause: Before a switchover, the loopback call was set up with 2:xx:xx:x:x:x (vbsc1's pkt port mac address) for both src and dst MAC address. After switchover, from packet capture, we found that the SRC MAC address= 2:xx:xx:x:x:xx (vsbc2's pkt port mac address). As a result, using the srcMac and dstMac caused a media issue.

Steps to Replicate: Use the following configurations to test:

• Set up loopback and normal calls for both SBC HW and SWe platforms.
• Do port and SBC switchovers.
• Use single and alternate media IP addresses on the loopback ports.

The code is modified to use loopback flag mirrored to standby BRM, and overwrite both srcMac and dstMac if loopback flag is set to true. This is done only on SWe when enabling or modifying the RID.

Workaround: No workaround.

In the case of delayed offer, the SBC changes the attribute “a=sendonly” into “a=recvonly” before relaying the 200 OK INVITE.

Impact: The direction attribute in 200OK sent in response to late media reinvite is set to an incorrect value when the direction attribute received in 200OK from other side is a value other than a=sendrecv.

Root Cause: The datapathmode on the side receiving the late media invite is incorrectly copied from the SBC local datapathmode from the answer leg PSP receiving 200OK when sendSbcSupportedCodecsInLateMediaReinvite flag is enabled.

Steps to Replicate: 

1. Enable the sendSbcSupportedCodecsInLateMediaReinvite on the TG towards UAC.
2. Send a direction attribute a=sendonly/recvonly/inactive from the UAS.
3. Send the 200OK to UAC should have direction attribute as expected (a=sendonly/recvonly/inactive respectively) and media is as per expectation.
   UAC---------------------------------SBC------------------------------------------------UAS
   INV(no sdp)->
   INV(a=sendrecv)->
   <-200OK(a=sendonly/recvonly/inactive)
   <-200OK(a=sendonly/recvonly/inactive)
   ACK(sdp)->
   ACK(no sdp)->

The code is modified so the datapathmode in the active PSP on late media offer side is recomputed when the sendSbcSupportedCodecsInLateMediaReinvite flag is enabled.

Workaround: Disable the sendSbcSupportedCodecsInLateMediaReinvite IPSP flag on TG receiving late media reinvite.

A switchover on the SBC 5400.

Impact: The SAM process core dumped when the SIP code was attempting to lookup a internal SIP Registration Control Block.

Root Cause: The core was the result of SIP code attempting to using a invalid index when accessing an array element for the purposes of finding a internal SIP Registration Control Block.

Steps to Replicate: The steps cannot be reproduced. 

The code is modified to prevent the use of an invalid index when accessing an array element.

Workaround: There is no known workaround.

When the call is dipped the invite sends only the LRN.

Impact: When ingress INVITE's DIVERSION header does not present and TO header is different from RURI, i.e., it has redirecting original number, the egress RURI will take translated number, i.e., LRN, as RURI in egress INVITE, after LNP dip.

Root Cause: The RURI should take LRN in egress, while DIVERSION header in ingress presents. But the code change for a previous issue has made RURI equal to LRN even DIVERSION header was missing.

Steps to Replicate: 

1. Set up SIP to SIP LNP call, on SBC and external PSX system.
2. The TO header username (phone number) is different from RURI.
3. There is no DIVERSION header.

This should reproduce the problem.

The code is modified the condition to only set RURI equal to LRN while DIVERSION header presents.

Workaround: None.

A customer's redundancy had an unexpected switchover.

Impact: The SBC is coring in the IPsec/IKE code when a call is using an IKE Protection Profile that was configured without any algorithms.

Root Cause: The SBC coredumped in the IPsec/IKE code due to an attempt to de-reference a NULL pointer. The pointer is NULL because the IKE Protection Profile being used was configured without any algorithms.

Steps to Replicate: 

1. Configure an IKE Protection Profile without any algorithms.
2. Run a call that will use this profile.

The code is modified to check for a NULL pointer and take the correct error path when the pointer is NULL.

Workaround: When configuring the IKE Protection Profiles, ensure that an algorithm is configured for this profile.

There was a SCM core dump.

Impact: A rare race condition triggers a bug that caused SCM to core.

Root Cause: A bug in the code causes an attempt to access freed memory. This Gateway-to-Gateway bug has has existed for a very long time, but was not encountered until now.

The bug is triggered by a rare race condition.

Steps to Replicate: This bug was found through code inspection.

The bug is triggered by a rare race condition that is not reproducible.

The code is modified to avoid accessing freed memory.

Workaround: There is no workaround.

The SBC is not passing an UPDATE to Egress.

Impact: If the SBC receives two 180 responses (first unreliable and second reliable) both with same SDP then the SBC is not sending UPDATE to Egress.

Root Cause: The SBC receives two 180 responses (the first unreliable and the second reliable) both with same SDP, if the SBC sends out SDP using the non reliable 180, the offer/answer state is not updated when receiving the second 180 based on how the SBC is coded today.

Steps to Replicate: 

1. sendSdpInSubsequent18x disable.
2. sendSdpInSubsequent18x enable.
   Change SDP in first nonreliable 180 and second reliable 180. In this case SDP gets queued however 180 ringing sent out to Ingress has current SDP in
   this case and offeranswer is completed.
3. Same SDP in first nonreliable 180 and second reliable 180 in this case offeranswer state completed.

The code is modified so that when receiving an SDP in unreliable (100rel not supported) 18x message, the Offer/Answer state does not update properly. If the SBC receives a subsequent 18x with 100 rel, the Offer/Answer state changes and this helps in correctly processing the subsequent UPDATE message.

Workaround: None.

A user is Registered on UDP and INVITE (same port/IP) on TCP fails with 403.

Impact: A user is Registered on UDP and INVITE (different port) on TCP fails with 403

Root Cause: Once the maskportforRcb flag is enabled, parent RCB is found but child RCB is not. Mask port and mask IP were not considered during child creation.

Steps to Replicate: 

1. Enable the maskportforRcb Flag and on ingress side set require registration to required.
2. Send register and receive 200 OK with p-Associated URI (To create child RCB) on UDP.
3. Send an INVITE with child user on TCP without specifying the port.
4. Verify that the RCB is found and the INVITE is routed properly.

The code is modified to address the issue.

Workaround: Not available.

The SIPRec recording failed for an EGRESS call with GCID.

Impact: The SIPREC sessions fail with the following "MAJOR" logs when a CS call is going for call a modification with re-INVITE and a corresponding re-INIVTE is triggered for this towards the SRS while a previous SRS SIP transaction is pending.

159 08192021 154751.374314:1.02.00.26912.MAJOR .SIPSG: sipsgRec.c (-5062) 529606794. [SipSgSendSRSMetadataUpdateCmd] SipSgSendSdpCmd Failed with status 491
185 08192021 154751.375973:1.02.00.26913.Minor .SIPSG: SIPRec Recording failed for EGRESS call with GCID 291518377 for Recorder 10.xx.xxx.xx:xxxx. Trunkgroup of Recorder: NICE_PRI_TG

Root Cause: The SBC always attempted to send another offer towards SRS even when a offer-answer was pending and this resulted in SRS session failure.

Steps to Replicate: Use the following setup to reproduce the issue.

1. Main Call (CS) session established.
2. SIPREC session (RS) is established.
3. CS call goes for modification with re-INVITE (Hold/Codec change).
   A SIPREC re-INVITE is triggered (based on CS re-INVITE).
4. The SRS does not respond for SIPREC re-INVITE.
5. Before the SRS answers the re-INVITE, a CS call triggers yet another modification with re-INVITE (unhold/codec change).
6. The SBC attempts to trigger SIPREC RE-INVITE again even when the previous transaction is pending and results in SRS session failure.

The code is modified so that when a SIP Offer-Answer towards SRS is in progress, the SBC does not attempt to send another offer, and instead queues the latest event until the current offer-answer is complete.

Workaround: None.

Details on "LAST RESTART REASON" under command "show table system serverStatus" are not updated correctly.

Impact: LAST RESTART REASON is not updated with proper reason of last restart.

Root Cause: Reading LAST TESTART REASON function is called twice, and it is set to default in the second call. 

Steps to Replicate: 

1. Check the LAST RESTART REASON from the CLI show table system serverStatus.
2. Call a different type of restart and verify if the LAST RESTART REASON is set properly or not.

The code is modified to read the LAST RESTART REASON only once. 

Workaround: None.

The SBC sending 481 for CANCEL in dialog transparency.

Impact: When the dialog transparency is enabled and call loops back to the SBC, the SBC does not handle a CANCEL sent from ingress endpoint.

Root Cause: The SBC expects a CANCEL to have callinfo header.

Steps to Replicate: 

1. Reproduce the issue.
   Dialog Transparency is enabled and the Call Loops back In.
   After adding PRACK, Ingress sends CANCEL without callinfo header.
   The SBC sends a 481.
2. With a fix, the SBC sends 200OK for CANCEL and relays to egress leg.

The code is modified to ensure the SBC finds the correct SG to handle the CANCEL even when the CANCEL does not have callinfo.

Workaround: None.

There was a coredump during a reboot on a customer SBC. 

Impact: A misconfiguration of GW Signaling Ports, in which there is a Secondary GW Signaling Port configured but no Primary GW Sig Port configured, and can lead to a core and switchover.

Root Cause: When no Primary GW Signaling Port is configured but a Secondary Signaling Port is configured, the SBC attempts to dereference a NULL pointer (pointer to Primary Sig Port) while attempting to get the DSCP value to use for the socket.

Steps to Replicate: 

1. Configure an SBC with a Secondary GW Signaling Port without configuring a Primary GW Signaling Port.
   (i.e. gwSigPort of SBC1 is in primary mode and SBC2 is in secondary mode)

2. Send a GW-GW call from a remote GW through this SBC using the address of the Secondary GW Signaling Port on an SBC.

   Without the fix, a core may occur.
   With the fix, no issue occurs.

The code is modified to prevent it from attempting to dereference a NULL pointer (pointer to Primary Sig Port) when it is looking up the dscp value to use for the socket.

Workaround: The workaround is to avoid configuring a Secondary GW Signaling Port without configuring a Primary GW Signaling Port.

The logs are getting printed in openclovis.

Impact: If condition present, SCM is filling openclovis logs with this error message:
nprintf buffer too small. Need 306 Have 300 File: /sonus/p4/ws/release/sbx5000_V08.02.02R001/marlin/SIPSG/sipsgRegAgent.c Line: 2527

Root Cause: The error message is logged because the code is attempting to write a debug log into a local buffer that is not big enough.

Steps to Replicate: This is no risk with/without fix.

The code is modified to increase the size of the local buffer.

Workaround: There is no workaround.

The Min-SE header is getting added to ingress metaDataProfile even though it is not in the initial INVITE.

Impact: For the SIPREC, Min-Se and Session-Expires headers are added to the metaDataProfile even though it is not received in the initial INVITE. 

Root Cause: When Incoming without support Timer, the SBC is incorrectly inserting a Min-Se and Session-Expires headers as part of an incoming INVITE.

Steps to Replicate: 

1. Configure the SIPREC setup and Min-SE and Session-Expires in sipRecMetadataProfile.
2. Having incoming INVITE without support timer and no Min-Se, no Session-Expires.

The code is modified to reset the value of the Min-Se and Session-Expires when received.

Workaround: SMM deletes the Metadata in the SIPREC INVITE. 

PortFix SBX-103228: Excessive logging and fast log rolling observed even at MAJOR level

Impact: When testing with Registration CAC enabled, this message is written aggressively per call, and contributes to high disk I/O and ENM processing.

MAJOR .SIPFE: *SipFeEpCacUpdateHpcValues:1506 peer was not static returning.

Root Cause: The message is written at the MAJOR level per call causing frequent log rolling.

Steps to Replicate: Enable registration CAC and make a call. This message is logged for every call and is basically useless at the MAJOR level.

The code is modified so the log event is moved to the INFO level.

Workaround: None.

PortFix SBX-115476: The SBC fails to send ACK in the re-INVITE callflow when E2E ACK and E2E re-INVITE flags are enabled.

Impact: The E2eAck is not working after an INVITE with replace is sent.

Root Cause: Currently, E2eAck configuration is applicable only on egress leg. After sending an INVITE with replace (ingress legC), the legC does not support e2eAck.

Steps to Replicate: 

1. Have A call B, C replace A.
2. C sends a re-INVITE to B.
3. The nAck from C fails to trigger the SBC to send Ack to B.

The code is modified so after sending a PSX query, if e2e re-INVITE is enabled, then turn on e2eAck flag as well.

Workaround: None.

Automatic AD Sync was not completing properly.

Impact: Automatic AD Sync was not completing properly.

Root Cause: During the change from Daylight Saving Time in the fall, the addMinsToTime routine does return a time the is greater then the time passed in. Consequentially, the code loops calling addMinsToTime expecting the time returned by addMinsToTime to be greater than the current time.

Steps to Replicate: 

1. Configure AD sync.
2. Set the time to 1:00 AM on the day of the current year where Daylight Saving Time ends.
3. After 2:00 AM on that day, turn on info level logging.
4. Observe the following message in the debug logs:
   DAP_CONFIG : Performing Sync Interval AD Sync Success
   appears.

The code is modified to return a time greater than the time passed when switching from Daylight Saving Time.

Workaround: None.

Portfix SBX-113242: The SLB/SBC is not fetching the correct branch param from merged VIA header.

Impact: Due to merge VIA headers, the SLB/SBC was not getting the correct instance id and response messages were not being routed to backend SBCs.

Root Cause: If the SLB/SBC receives a VIA header as merge of two VIA headers, it was overwriting the first via branch param from the second VIA branch param, and due to this SLB was not getting the correct instance id.

Steps to Replicate: 

1. UAC will send an INVITE towards the SLB/SBC.
2. The SLB/SBC sends the INVITE towards UAS.
3. UAS will send 183 response messages with merged the VIA headers (including received param) after From and To header towards the SLB/SBC.
4. The 183 should be routed to the SBC and then towards UAC.

The code is modified to set the SLB/SBC read only in the first branch param, that is what we need from top VIA and ignore the rest branch param.

Workaround: None.

Portfix SBX-111731: A restart on the configuration for linux service in systemd unit file is wrong.

Impact: The services intervalstats, trc-anonymization continues restarting on failure (if failure happens within 360 seconds).

Root Cause: The restart configuration for the linux service in systemd unit file is wrong.

Steps to Replicate: Stop the process services intervalstats, trc-anonymization five times and do not retried.

The code is modified to address the issue. 

Workaround: None.

PortFix SBX-102598: There were MFG Intermittent test failures.

Impact: Xaui link is not always coming back up after enabling/disabling the internal loopback mode.

Root Cause: Xaui link is not always coming back up after enabling/disabling the internal loopback mode.

Steps to Replicate: Run the ESS test mode for days.

The code is modified to the enable/disable loopback code.

Workaround: None.

PortFix SBX-110245: The AddressSanitizer: heap-use-after-free /sonus/p4/ws/jenkinsbuild/sbxAsan100/marlin/SIPSG/sipsgMsgProc.c:7097 in SipSgProcessNrmaUpdateNfy(SIPSG_CONTEXT_STR*, nrma_update_nfy_msg_str*)

Impact: The ASAN detected "AddressSanitizer: heap-use-after-free" while accessing SG_CCB_STR pointer.

Root Cause: The SBC was trying to access the SG_CCB_STR pointer that is already been freed.

Steps to Replicate: Run a basic call where ICID value of P-Chargingector header in INVITE message is NULL or absent.

The code is modified to check the pointer is null or not before accessing it.

Workaround: None.

PortFix SBX-109059: The SIPSG FAX multiple streams initiated fax call fails (muted T.38 and audio)

Impact: The multi-stream fax call is failing with muted T38 and audio.

Root Cause: There is a mismatch in the media streams of the SBC offer and the peer answer. Due to this, the images media stream received from UAS is mapped incorrectly to the audio stream in the SBC offer. This is happening when advertise audio only flag is enabled.

Steps to Replicate: 

1. UAC sends an INVITE with Audio and Image.
2. The SBC sends an INVITE with Audio to UAS due to Advertizeaudio flag is enabled.
3. UAS responds 200 OK with Audio and call gets connected as audio.
4. UAS sends re-INVITE with Image.
5. The SBC sends a re-INVITE with both Audio and Image muted causing call teardown.

The code is modified so the SBC maps the media streams in offer and answer correctly when advertise audio only flag is enabled.

Workaround: None.

Portfix SBX-110215: A coverity issue.

Impact: Code fix for an SCM coredump (SBC-108237) when the Invite gets a transaction timeout and the 200 Ok for Invite is received at the same time needed some cleanup. Coverity issue: CID:338630

Root Cause: In the problem scenario, the SBC is trying to send ACK for the 200 OK, when the SCM cores while creating a SIP Transaction for ACK Message.

Steps to Replicate: 

1. Run a Basic Invite call flow.
2. Do not answer for Invite on the UAS side.
3. After 32 seconds, send 200 OK for Invite from the UAC side.

A race condition might occur and result in coredump.

The code is modified not to send ACK in this scenario as the CseqType and CseNumber are unknown and 0 respectively.

Workaround: None.

PortFix SBX-102925: There are issues in the Ova\QCow2 installation.

Impact: Differing prompts and root passwords are created for the SBC SWe installed through the ISO, and the SBC SWe created through the image launch method, 

Root Cause: While installing through the image launch method, the SBC SWe was configured with the Cloud SBC method of creating the root password and prompt.

Steps to Replicate: 

1. Install the SBC SWe using ISO. Check the root password and CLI prompt.
2. Instantiate the SBC SWe using Image launch. Check the root password and CLI prompt.

Results from step 1 and 2 should be same.

The code is modified to ensure both image launch method and ISO install method of the SBC SWe has no root password and the same CLI prompt.

Workaround: A user can manually remove root password on ISO installed SWEs using the command below:
usermod root -p '*'

PortFix SBX-104319: Conflicted design between two features “sdpRelayAttribute” and “Send RTCP Bandwidth Info”.

Impact: The SBC sends duplicate b=RR and b=RS values when “sdpRelayAttribute” and “Send RTCP Bandwidth Info” are enabled together.

Root Cause: The SBC was not honoring "sendRTCPBandwidthInfo" only when “sdpRelayAttribute” was enabled with TFT. Due to this, the duplicate of b=RR and b=RS was seen, when “sdpRelayAttribute” was enabled without TFT.

Steps to Replicate: Configuration:

1. Enable sdpAttributesSelectiveRelay flag under TG.
   set addressContext default zone ZONE_V6 sipTrunkGroup TRUNK_V6 media sdpAttributesSelectiveRelay enabled
   set addressContext default zone ZONE_V4 sipTrunkGroup TRUNK_V4 media sdpAttributesSelectiveRelay enabled
2. Enable the RTCP with below settings.
   set profiles media packetServiceProfile DEFAULT rtcpOptions rtcp enable rrBandwidth 500 rsBandwidth 150

Procedure:

1. Make a basic call so that the bandwitdth parameters are received from the endpoint as b=RR:1125, b=RS:775.

Expected Result:

1. The SBC should transparently send the b=RR & b=RS parameters in SDP and should not honor the bandwidth configured in the PSP.

Also, the SBC should not add duplicate b=RR and b=RS parameters in the SDP honoring the configured bandwidth values in the PSP.

The code is modified so if “sdpRelayAttribute” is enabled separately without TFT, and "sendRTCPBandwidthInfo" is enabled, then the SBC does not honor "sendRTCPBandwidthInfo" and does not send a duplicate value of b=RR and b=RS.

Workaround: None

PortFix SBX-70800: AWS: Observing that metaVariable table is getting modified on loading the backup config file of one instance in other instance.

Impact: When loading the backup configuration from one SBC instance to another, the metavar table is getting populated with the list of metavars from both instances. This by itself does not cause any issues so long as the metavars are unique, its just confusing to see the metavars for another instance in the table.

Root Cause: The code was not removing the existing metavars prior to loading the configuration of another instance. This meant the metavar table had both sets of information.

Steps to Replicate: 

1. Create a backup configuration of one cloud instance (instance1).
2. Loaded the backup config file into a new cloud instance (instance2).
3. Check the metaVariables table and see if it contains the metavars from both instances.

The code is modified to flush the metavars for the existing instance prior to loading the configuration from another instance.

Workaround: None.

PortFix SBX-111407: The SBC DRBD mount was not visible on the active SBC. 

Impact: The SBC DRBD mount was not visible on the active SBC.

Root Cause: When the sbxrestart is issued from platform manager, the DRBD gets mounted on apache's mountspace.

Steps to Replicate: 

1. Bring up both the nodes.
2. Run the SBC restart from pm on active.
3. The new active must have the DRBD mounted.

The code is modified to address the issue.

Workaround: None.

PortFix SBX-106854: The ASAN runtime flatbuffer serialization unit test fails with asan=1.

Impact: The ASAN build is failing due to automated test failures related to serialization.

Root Cause: Automated unit test is failing because of uninitialized variables.

Steps to Replicate: Create an ASAN build and it should be successful.

The code is modified to initialize the variables that were causing the Unit tests to fail.

Workaround: None.

There was an issue regarding field Ingress Codec inside CDR for some calls with CLEARMODE.

Impact: A SIPI Invite call fails when CLEARMODE as CODEC feature is enabled.

Root Cause: The call failed as clearmode in PSP was becoming NULL.

Steps to Replicate: CLEARMODE as CODEC enabled and PSP has CLEARMODE as entry.
Example: set addressContext default zone ZONE2 sipTrunkGroup CORE_ZONE signaling clearModeAsCodec enabled

Test steps:

1. Use the SIPI scripts.
2. Enable ClearModeAsCodec flag in both ingress and egress TG. Add clearmode codec in ingress and egress PSP.
3. Use the scripts and make a SIPI-SIPI call.
4. A call should connect and the CDR should be populated as DATA, CLEARMODE (P:71:0).

Tests can be done to verify different call flows:

1. A SIPI-SIPI call with ingress offers CLEARMODE , Egress responds CLEARMODE.
   expected output: CDR and Call media status cli command should show data and CLEARMODE.
2. A SIPI-SIP call with ingress offers CLEARMODE , Egress responds CLEARMODE.
   expected output: CDR and Call media status cli command should show data and CLEARMODE.
3. A SIPI-SIP call with ingress offers CLEARMODE + G711U , Egress responds G711U.
   expected output: CDR and Call media status cli command should show audio and G711U.

The code is modified so that when a CLEARMODE as CODEC enabled and SIPI with CLEARMODE is received, a call is processed and the CDR gets updated with CLEARMODE.

Workaround: Not available.

The SecGetTlsProfileDataByName() selects a TLS profile other than the configured one.

Impact: The SBC uses the wrong TLS profile.

Root Cause: The 'lookup by name' function returned with the wrong profile.

Steps to Replicate: 

1. Enable DBG INFO level logging.
2. Create two TLS profiles as described in the example below:
   set profiles security tlsProfile TEST_ACCESS cipherSuite1 rsa-with-aes-128-cbc-sha
   set profiles security tlsProfile TEST_ACCESS allowedRoles server
   set profiles security tlsProfile TEST_ACCESS authClient false
   set profiles security tlsProfile TEST_ACCESS v1_0 enabled
   commit
   set profiles security tlsProfile TEST cipherSuite1 rsa-with-aes-128-cbc-sha
   set profiles security tlsProfile TEST allowedRoles clientandserver
   set profiles security tlsProfile TEST authClient false
   set profiles security tlsProfile TEST v1_0 enabled
   commit
3. Assign a second profile to one of the sipSigPort.
4. Send PDU through the sipSigPort used in Step 3.
5. Check DBG log and look for error messages and compare the profile ID assigned in Step 2.

The code is modified for both TLS and DTLS profile lookup by name routines to return the requested profile.

Workaround: None.

PortFix SBX-101409: A T140 call - one-way stream - zero media port (NAPT media).

Impact: The call ends up in one-way audio after the called party puts the call on hold and off hold twice.

Root Cause: As a result of call-modify a couple of times, the RTCP NAPT learning completes before RTP NAPT learning.

This results in RTCP Remote Address being updated, which has remote RTCP Port.

Due to incorrect code in RTCP modify flow, remote RTCP port, gets assigned to RTP port. This results in one-way media.

Steps to Replicate: 

1. Run basic SIP to SIP call with NAPT & RTCP enabled.
2. Hold/Unhold the call a few times to check for proper 2-way audio.

Set the correct RTP Port as part of RTCP modify flow.

Workaround: Since the issue is caused to RTCP NAPT learning completed before RTP during multiple hold/unhold scenario, a workaround could be:

1. Disable RTCP or
2. Disable NAPT.

PortFix SBX-104253: The "Deleting the a=maxptime" SMM is not working after an SBC restart.

Impact: SMM with sdpContent not getting applied after restart.

Root Cause: At time of restart CPX trying to read SMM with sdpContent from the wrong path.

Steps to Replicate: 

1. Spawned an HA setup.
2. Configured SMM rule with sdpContent and run the scenarios.
3. Switchover the system and re-run the scenario.
4. Switchover again and re-run the scenario.

The code is modified to address the issue. 

Workaround: Delete the SMM rule and configure again.

PortFix SBX-105637: The AddressSanitizer: detected heap-buffer-overflow on address 0x61b000013128 at pc 0x55e3eea0419e bp 0x7ff7a52768a0 sp 0x7ff7a5276898 in ScmProcess_0.

Impact: There was invalid memory access when the SBC receives the 500 Internal Error to REGISTER.

Root Cause: The root cause of this issue is accessing invalid memory while accessing callData, trying to read off the data from the end of memory block. It can potentially cause the coredumps if the memory block is at the edge of the accessible memory region.

Steps to Replicate: Testcase:
Description:

1. Clean up SAs if unexpected response received

Procedure:

1. Monitor the exchange.
2. Send an initial reg message.
3. Receive the 401 challenge.
4. S-CSCF responses with 500.

Expected Results:

1. Verify IPsec SA's deleted (ip xfrm state).

The code is modified to access the respective application data (It can be call data, registration data or subscription data) based on type of SIP message.

Workaround: None.

PortFix SBX-105763: Move the dmesg monitoring from SM to a platform cron job

Impact: Move dmesg monitoring from SM to a platform cronjob

Root Cause: Since the dmseg can be large for long-running SBCs (thus, take longer to dump logs), the function can take longer causing an SM healthcheck.

Steps to Replicate: Check for "/var/log/sonus/tmp/dmesgErrorMarker.tmp" after manually running the script

The code is modified to run every minute as a cron job to find i/o and filesystem errors in dmesg. If an error is found, the SBC creates a marker file in tmp, which is later used by other scripts to check sanity of the system.

Workaround: None

PortFix SBX-107798: The one way audio when forked leg is MS Teams without ICE/NAT.

Impact: For an incoming call that is forked to multiple destinations and DLRBT is enabled, there is a possibility of audio flowing only in one direction when the call gets established.

Root Cause: When a 180 ringing is first received from one of the forked destinations, this triggering the SBC to play ringback tone, but if the call subsequently receives 18x messages and RTP media (for media cut through) from a different destination, the SBC fails to activate the RTP media flow resources correctly at the ingress leg of the call because these resources are already activated for playing the ringback tone for the other fork.

Steps to Replicate: Set up
---------
The SBC configured with call forking such that call from ingress (A) to be forked to egress (B) and egress (C).
The DLRBT should be enabled on ingress and egress trunk toneAndAnnouncementProfiles.

Procedure
--------------

1. Initiate the call with an INVITE from A that should be forked to B and C.
2. From the egress endpoint B respond with 180 and then from egress endpoint C respond with 180.
3. From egress endpoint C respond with 183 with SDP.
4. Send the RTP media packets back from C to cause media CUTTHRU.
5. From egress endpoint C respond with 200 OK to connect the call.

Expected Results
-----------------------
1. The INVITE sent to B and C.
2/3. The SBC sends back 180 towards A and starts to play ringback tone towards A.
4/5. The call is established, the SBC sends CANCEL towards B, ringback tone should stop and media should flow in both directions between A and C.

Before the fix, media was not flowing from A to C.

The code is modified to re-activate the media resources after the call is answered on one of the forks and the remaining forks have been cleared.

Workaround: Not available, but the issue does not occur if the DLRBT is not enabled.

Portfix SBX-107973: The SBC adding RR and RS attributes twice in the egress INVITE when multiple m lines present in SDP.

Impact: The SBC was adding RR and RS attributes twice in the egress INVITE when multiple m lines present in a SDP.

Root Cause: The SBC is not setting the default value of M lines present in SDP when number of lines is greater than 1 and as a result, the SIPS value is getting added.

Steps to Replicate: Configure these:
set profiles media packetServiceProfile DEFAULT rtcpOptions rtcp enable

set profiles signaling ipSignalingProfile DEFAULT_SIP commonIpAttributes flags sendRTCPBandwidthInfo enable
set profiles signaling ipSignalingProfile DEFAULT_SIP commonIpAttributes flags sendRtcpPortInSdp enable
comm

set addressContext default zone ZONE_EGRESS sipTrunkGroup TG__EGRESS media multipleAudioStreamsSupport enabled
set addressContext default zone ZONE_INGRESS sipTrunkGroup TG_INGRESS media multipleAudioStreamsSupport enabled
comm

An incoming INVITE has multiple Audio line with:
b=RS:250
b=RR:250

The code is modified so that if value is set to default and it is not getting modified, the SIP does not send the RR and RS value twice.

Workaround: When the sdpAttributesSelectiveRelay is enabled, the SBC does not send RR and Rs twice.

PortFix SBX-108029: The SIPSG_MSG_PARAM_SIPSG_REDUND_ACT_STR is not registered for serialization correctly before 10.0 release

Impact: After LSWU, some of the event record fields are not serialized properly for the calls related to Register and OOD messages.

Root Cause: Event record accounting details are not registered for Serialization.

Steps to Replicate: 

1. The SBC is an older release(< 9.2.1).
2. Configure to generate event record for Register.
3. Make a Register call.
4. Send a Refresh register.
5. LSWU to 9.2.1.
6. Complete the registration call.
7. Check the details of generated event record.

The code is modified to address the issue.

Workaround: None.

Portfix SBX-108572: [ASAN]: /localstore/ws/jenkinsbuild/sbxmainasan/marlin/SIPS/sipsParseActions.c:12698:76: runtime error: index 20 out of bounds for type 'sip_nameval_str [20]'

Impact: When the PUBLISH message is received with 20 params in the Contact Header, the SBC throws a runtime error: index 20 out of bounds for type 'sip_nameval_str.

Root Cause: The param check for boundary condition was missing while parsing the contact header. While it was checking the params, the number of params should be less than 20, but the condition to handle number of params is not specified.

Steps to Replicate: Send a PUBLISH message with 20 or more params in the Contact header.

The code is modified so if the number of params is 20, the SBC should throw a parse error.

Workaround: None.

PortFix SBX-108410: [ASAN]: sanitizer.CE_2N_Comp_ScmProcess_3.8866:==CE_2N_Comp_ScmProcess_3==8866==ERROR: AddressSanitizer: heap-use-after-free on address 0x6190001c77dd at pc 0x558bcc9ff877 bp 0x7fea305f4e00 sp 0x7fea305f45b0.

Impact: The ASAN reported a "AddressSanitizer: heap-use-after-free" error when Subscribe request having a NULL character in quoted string.

Root Cause: Invalid access of the freed memory occurred. Accessing memory after it is freed can cause unexpected behavior that may result in coredumps.

Steps to Replicate: The codenomicon subscribe-notify suite.

The code is modified to now log a parser error.

Workaround: None.

PortFix SBX-108616: Fora  Late Media Call, the SBC is not sending a second UPDATE towards ingress when the DLRBT is enabled.

Impact: In a late media convert call scenario, when the DLRBT is enabled, the SBC is not sending UPDATE towards the ingress with the list of codecs received from UAS.

Root Cause: The minimizing of media changes from other call leg functionality is suppressing the triggering of the UPDATE towards UAC.

Steps to Replicate: 

1. The UAC sends INV without SDP to the SBC.
2. The SBC sends INV with PCMA PCMU G729 101.
3. The UAS sends 180 with PCMA G729.
4. The SBC sends 180 with PCMA G729.
5. The UAC sends PRACK with G729.
6. The SBC starts playing tone with G729 towards ingress EP.
7. The UAS sends 200 OK (for INV) without SDP.

The code is modified to identify this particular call scenario, such that, the SBC sends an UPDATE towards the UAC if the tone codec is different from the end to end negotiated codec.

Workaround: None.

Portfix SBX-108619: [ASAN]: /localstore/ws/jenkinsbuild/sbxmainasan/marlin/SIPS/sipsPreParse.c:1742:46: runtime error: signed integer overflow: 2002002002 * 10 cannot be represented in type 'int'.

Impact: A runtime error is seen in the SAM process. When an INVITE is sent out and response code received is very large, we will see the following issue:
Runtime error: signed integer overflow: cannot be represented in type 'int'

Root Cause: When the SIP Response code is very large, there is a signed integer overflow during the processing of the SIP PDU.

Steps to Replicate: An INVITE is sent out to the egress. If the response code received is very large, the issue is seen.

The code is modified so if the SIP response is greater than or equal to max response code, the SBC throws an error.

Workaround: None.

PortFix SBX-109187: [ASAN]: sanitizer.CE_2N_Comp_ScmProcess_0.32472:/localstore/ws/jenkinsbuild/sbxmainasan/marlin/SIPS/sipsParseActions.c:16242:70: runtime error: null pointer passed as argument 2, which is declared to never be NULL.

Impact: The NULL pointer was accessed during the codenomicon test with an ASAN SBC build.

Root Cause: The Codec Attribute has been accessed in the SDP without a proper NULL check.

Steps to Replicate: 

1. Install the ASAN build on the SBC.
2. Run Codenomicon INVITE response with the outgoing Bye suite.

The code is modified to add the defensive NULL check.

Workaround: Not applicable.

Portfix SBX-109412: The destination buffer was too small during the snprintf function.

Impact: During the snprintf function call, the destination buffer was not big enough to copy the source string.

Root Cause: The size of the destination buffer was smaller than the source buffer because the destination buffer length check was not present.

The buffer file size should be 256 bytes. File: call/sonus/p4/ws/jenkinsbuild/sbxAsan100/marlin/SIPS/sipsParse.c

Steps to Replicate: 

1. When the callId length was more then the defined max callId length this error was seen.
2. Run a call with callId length greater than 256 bytes. This error should not come.

The code is modified to address the issue.

Workaround: None.

Call to landline number was incorrectly listed as busy.

Impact: The SBCs try to map the non-E164 display name of p-asserted-Identity TEL to generic address for called user.

Root Cause: When the SBCs received both pai (tel) and pai (sip) headers, it attempts to treat as a Japan ttc even though the display name in pai(tel) is not E164 format.

Steps to Replicate: Run the following command string to replicate the issue:

config
sipTrunkGroup IAD signaling callingPalingParty enabled
Incoming msg has:
P-Asserted-Identity: "12345 test user " <tel:+1xxxxxxxxxx;rn=214960>, "12345 test user" <sip:+1xxxxxxxxxx;rn=214960@test.com>

outgoing INVITE
From, contact userpart is 12345

Validate if the display name is E164 format, then try to map display name as generic address for called user to address the issue.

Workaround: Use the SMM to delete incoming display name in pai(tel) header.

For an outgoing message, add it back if pai is config transparency.

PortFix SBX-109443: ERROR: The AddressSanitizer: detected heap-use-after-free on address 0x61900412bbce at pc 0x55d0b8c48037 bp 0x7fb50a457b00 sp 0x7fb50a4572b0 READ of size 2 at 0x61900412bbce thread T11.

Impact: The ASAN reported "AddressSanitizer: heap-use-after-free" error for subscribe message received with Proxy-Authorization header having auts parameter.

Ex: Proxy-Authorization: Digest auts*="0x01P+20"

Root Cause: An invalid access of the freed memory occurred in this case. Accessing memory after it is freed can cause unexpected behavior that may result in coredumps.

Steps to Replicate: Send a SUBSCRIBE message received with Proxy-Authorization header having auts parameter.

The code is modified to address the issue.

Workaround: None.

Portfix SBX-109862: The SBC is not rejecting the SRTP call when disallowSrtpStream is enabled.

Impact: The SBC is not rejecting the SRTP call when disallowSrtpStream is enabled in ingress PSP and Invite is received with only SRTP stream.

Root Cause: This is specific call flow when disallowSrtpStream is enabled and an INVITE is received with only SRTP stream. This scenario was not handled and the SBC was not rejecting call.

Steps to Replicate:

1. Feature flag "multipleAudioStreamsSupport" under trunk group media is enabled for both ingress and egress.
2. A disallowSrtpStream flag under trunk group should be enabled on the ingress side.
3. Make a call with two SRTP stream.

The code is modified so that when the disallowSrtpStream is enabled and an INVITE is received with only an SRTP stream, the call is rejected with 488.

Workaround: None.

Portfix SBX-110291: AddressSanitizer: The ASAN detected a heap-buffer-overflow-SipSgIncomingCallNfy (unsigned int, sip_addr_str*, sip_msgbody_str*, sip_options_str*, unsigned int, unsigned int, bool) /sonus/p4/ws/jenkinsbuild/sbxAsan100/marlin/.

Impact: The ASAN detected "AddressSanitizer: heap-buffer-overflow" while copying the contents of the SIP-I base version string internally.

Root Cause: The SIP code was always copying a fixed amount of memory when reading the SIP-I base and version strings to internal memory.

Steps to Replicate: Run a SIP-I call flow with INVITE and CANCEL messages.

The code is modified to copy exactly string length size of the SIP-I base and version content to avoid reading past the end of memory buffers as this can cause coredumps.

Workaround: None

Portfix SBX-110640: The CDC config not replicated to standby during SWO when there are no active calls.

Impact: In a PC2.0 setup with HA, when a switch over is performed without LI calls then the interception stops working on the new active.

Root Cause: The standby SBC is not processing CDC config for realm2hash. Realm2hash is only created if SBC has active LI calls at the time of transition from Active to standby. In case when SBC don't have Active call realm2hash is not created on transition from Active to standby.

Steps to Replicate: 

1. Configure the SBC for LI PC2 flavor call.
2. Disabled the peer and realm.
3. Enable peer and realm.
4. Check the realm2hash entry on current active.
5. Perform switch over from active to standby.
6. Check the realm2hash entry on current active.

Standby code is updated to process CDC config and create realm on standby.

Workaround: Disable/enable CDC Diameter Realm Route and Peer on the new-active.

PortFix SBX-110665: Internal IP peers blacklisted after switchover despite that OPTIONS ping worked fine.

Impact: When the system experiences a Split Brain condition, the ipPeer(s) using a pathCheck may become BLACKLISTED and may not be RECOVERED.

This can result in some ipPeer(s) becoming BLACKLISTED permanently.

Root Cause: During a Split Brain, both CEs are running in ACTIVE state. In the error case, the PATHCHK sends BLACKLIST event(s) to the ARS on both systems (CEs). This can result in a PATHCHK and ARS being out-of-sync with respect to the ipPeer(s) BLACKLISTED/RECOVERED state.

Steps to Replicate: This issue is very difficult to reproduce. We have only been able to reproduce the issue by performing “kill-stop 2” on a KVM based system, that has a number of ipPeer(s) with pathCheck profile state enabled.

The code is modified so that the PATHCHK sends events to the ARS only on the system (CE) that is being executed. The PATHCHK is prevented from sending events to ARS on the other system (CE).

Workaround: If this issue occurs, the ipPeer(s) that are stuck on BLACKLISTED may be RECOVERED through the CLI by disabling the ipPeer(s) pathCheck state.
Customers can state disable and state enable the ipPeer(s) pathCheck state.

Portfix SBX-110755: The ACL rules configured with ipInterface are disabled on every SBC start/restart in SBC5400/10GB pkt configuration.

Impact: The IPACL rules created with ipInterface defined are not getting installed on the SBC 5400 systems when there is no license present, or after a license is added.

Root Cause: The NP will not install an IPACL rule with an ipInterface defined if that interface is not UP and active. The SBC will retry to install failed rules when it sees the port come up. In this scenario the timing is off, however, it tries too soon as the physical port is up but the associated ipInterface is not yet up.

Steps to Replicate: The test involves a SBC 5400 without a license for its 10G packet port. The IPACL rules with ipInterface defined on that packet port will then fail to install. Once the license is successfully added, the IPACL rules should be installed.

The code is modified to add an additional delay before attempting to reinstall the failed IPACL rules when the port comes up and to look for an additional failure that starts a timer and try again.

Workaround: Do not use IPACL rules with ipInterface defined.

PortFix SBX-111004: The SCM Process cores on the customer PSBCs PS40, PS41, and PS42.

Impact: A SCM Process coredump was observed because of Segmentation fault when a call was intercepted with PCSILI and media monitoring enabled.

Root Cause: The SCM Process dumped core while pushing request for splitter resources and media monitoring was enabled.

Steps to Replicate: 

1. Configure the SBC and PSX for basic audio call.
2. Configure the customer CLI on Egress Leg for basic call.
3. Enable Media Monitoring on Egress leg for basic call.
4. Enable the Tones on Ingress Leg.
5. Perform an Interception based on PCSI_LI.

The code is modified to avoid the coredump.

Workaround: None.

PortFix SBX-111177: The SBC incorrectly interprets RTCP packets as RTP when using DLRBT.

Impact: The RBT (ring back tone) can be terminated early when the DLRBT (dynamic local ring back tone) is enabled and RTCP packets or comfort noise packets arrive with the B-party.

Root Cause: When the DLRBT functionality was initiated it was not informed that RTCP could be received. This resulted in RTCP packets being treated as RTP and caused the SBC to think RTP was learned. Unexpected RTP comfort noise packets can also cause the SBC to think RTP was learned.

As soon as SDP was received in 183, the SBC triggered cut through and the RBT was stopped even though the call was not answered. This left the A-party with silence until the call was answered.

Steps to Replicate: Make a PSTN to MS Teams call with DLRBT enabled. Leave the call in ringing state with MS Teams for a long time and check that the ring tone is continually generated.

The code is modified to correctly identify RTCP packets and not use this as an indication to media being learned so that the ring tone continues to be played until real RTP packets arrive.

The learning mechanism is also modified to look for five or more RTP packets within a five second period to establish that RTP is learned. This is to assure that occasional unexpected comfort noise packets are not used for learning.

Workaround: If RTCP is not required on the egress leg then disable it. If RTCP is required there is no work around.

PortFix SBX-111277: The reason parameter was in the wrong History Info Index.

Impact: Egress History-Info header's index is incorrect in a SIP to SIP Call, when an ingress call with Diversion mapping to egress leg with History Info. In the egress IPSP the flag diversionHistoryInfoInterworking = disable by default, and flag includeReasonHeader = enabled. Other flags as below:
dataMapping diversion;
historyInformation {
includeHistoryInformation enable;
causeParameterInRFC4458 disable;
reasonWithCauseValueAsPerRFC4244 enable;
}

Root Cause: When the index of egress History-info header is generated in above condition, it's generated in a wrong way.

Steps to Replicate: 

1. Set up a SIP to SIP call.
   In the egress IPSP of the egress TG, set
   diversionHistoryInfoInterworking = disable
   includeReasonHeader = enabled.
   Other flags:
   dataMapping diversion;
   historyInformation {
   includeHistoryInformation enable;
   causeParameterInRFC4458 disable;
   reasonWithCauseValueAsPerRFC4244 enable;
   }
2. Make a SIP call. The Ingress INVITE contains a Diversion header.
3. Check the egress INVITE.
4. Enable the diversionHistoryInfoInterworking flag to compare the egress INVITE with the same ingress INVITE.
   

The code is modified to address the issue.

Workaround: None, unless set diversionHistoryInfoInterworking = enable.

PortFix SBX-111349: An SBC 7000 dual crash within 1 minute (old Active side - Fm, new Active - Scm).

Impact: The application on both active and standby switched over and the application reset in a short order of each other. Core files are created on both sides of the HA pair.

Root Cause: The issue occured during multiparty call processing where the SBC tries to determine a whether message was sent for the ingress or egress call segment. The code was accessing the pointer to the multi party call resource after it was freed up and set to NULL.

Steps to Replicate: This issue is not reproducible. Potentially due to a race condition in the code.

The code is modified so the multi party call pointer is not NULL before reading from it to avoid the coredump.

Workaround: There is no known workaround for this issue.

PortFix SBX-111659: The intercept is not working when P-Com.Session-Info is received in INVITE and tones are enabled.

Impact: When the lawful intercept target is specified in the P-com-session-info header of the INVITE message, the intercept does not occur if an announcement resource is applied to the call and then freed up. There is no issue if the P-com-session-info header is received in the backward direction.

Root Cause: This occurs when using P-early-media with tone profile and media monitoring profile applied to the call flow and the B-party sends P-early-media with a=inactive to start the media monitoring and does not provide RTP or does not send 200 OK prior to the monitoring timer expiring which results in the SBC playing RBT via an announcement resource. When the media path is later cut through and the announcement resource is released the intercept information was mistakenly freed up and the intercept did not occur.

Steps to Replicate: 

1. Run a lawful intercept call flow using P-com-session-info header in the INVITE to indicate the target intercept point. The call should be configured with P-Early-Media, tone profile and monitoring profile to check for RTP packets received.
2. In response to the egress INVITE, send back an P-early-media header with a=inactive and no RTP packets prior to the monitoring timer expiring so the SBC plays RBT.
3. Answer the call and check that media is being sent to the intercept server.
   

The code is modified to ensure the intercept information is not freed up when freeing the announcement resource.

Workaround: None.

Portfix SBX-11174: The SBC takes 72 seconds to send BYE to transferor after call termination.

Impact: The SBC is taking 72 seconds to send BYE to the transferor when the transferee disconnects before a transfer target accepts the call.

Root Cause: One of the disconnect events towards transfer target is getting ignored in the call control state machine. As a result, a release timer of 70 seconds is getting triggered later and the SBC is sending BYE towards transferor.

Steps to Replicate: 

1. A and B call is connected.
2. B sends a REFER to the SBC.
3. The SBC sends a new INV to transfer target C.
4. C sends 180 Ringing to the SBC.
5. A sends BYE to the SBC before C answers the call.

The code is modified so that the SBC sends a disconnect immediately towards the transferor.

Workaround: None.

PortFix SBX-111956: An early media case in SIPREC re-INVITE is not triggered with latest values for metaDataSource=fromLatest.

Impact: Call modification events like codec change/hold/un-hold on the CS that occur before the RS Session is established is not propagated to the RS Call (SIPREC Session).

Root Cause: The SBC does not attempts to trigger SIPREC re-INVITE with updated session characteristics when initial SIPREC INVITE transaction itself is pending with SRS and Main Call received a re-INVITE.

Steps to Replicate: Run the following procedure to recreate the scenario:

• Main Call (CS) session established.
• SIPREC INVITE (RS) is sent towards SRS.
• SRS does not respond for SIPREC INVITE. (SBC - SRS INVITE transaction is pending).
• CS call goes for modification with RE-INVITE. (Hold/Codec change).
• SRS answers the initial SIPREC INVITE after Main CS Call RE-INVITE.

The code is modified to queue the SIPREC RE-INVITE event when an initial SIRPEC INVITE transaction is pending.

When the initial SIPREC INVITE transaction is completed, the queued SIPREC RE-INVITE is sent towards the SRS (This contains the latest session characteristics).

Workaround: None.

PortFix SBX-112060: STIR/SHAKEN services were failing in the second dip in 2-stage call (SIP-SIP).

Impact: The SBC was not sending STIR/SHAKEN parameters, such as identity headers, to the PSX in the trigger request portion of a two-stage call or processing the STIR/SHAKEN parameters in the response.

Root Cause: The SBC was missing code to handle the interaction between two-stage calls and STIR/SHAKEN logic.

Steps to Replicate: Make a two-stage call where the INVITE contains identity headers and check they are sent to the PSX in both the policy request and trigger request.

The code is modified to send STIR/SHAKEN parameters such as identity headers to the PSX in the trigger request portion of a two-stage call and process the STIR/SHAKEN parameters in the response.

Workaround: None.

PortFix SBX-112117: There is a Wall LRA I-SBC switchover after the applying ACLs.

Impact: Configuring certain combination of ACLs may exceed the hugepage requirement than available in the system. This will cause the ACL configuration to fail and SWe_NP to exit.

Root Cause: The code did not account for available hugepages during an ACL configuration.

Steps to Replicate: Bring up setup in I-SBC profile. Apply the ACL configuration used by customer.

The code is modified to limit the hugepage memory use during an ACL build and ensure it is within the allocated limit.

Workaround: Use more than 64G RAM.

PortFix SBX-112267: Correct the 92x serialization code for a registration control block.

Impact: When the registration control block is made redundant and the active/standby instances are running different software releases e.g. during upgrade, the redundancy logic might not work correctly if the registration control block contains a P-Charging-Vector (PCV) and/or P-Charging-Function-Addresses (PCFA) information. The redundancy logic will work correctly post upgrade when serialization of the redundancy data is not required.

Root Cause: The parameter lengths of the PCV and PCFA redundant parameters was not calculated correctly. This can lead to the redundancy code being unable to process all the redundancy information correctly.

Steps to Replicate: Run registration related tests with PCV and PCFA header parameters in an older release and then upgrade to 922R2 or later.

The code is modified to correctly set the length of these parameters to avoid problems with redundancy.

Workaround: None.

Portfix SBX-112429: The NrmGetCallCount was returning max global callcount, instead of the current stable calls.

Impact: The I-SBC sends the total (cumulative) number of stable calls instead of current stable calls to the SLB in its utilization message. This could result in the SLB dropping messages if it thinks the I-SBC has reached 100% utilization.

Root Cause: The I-SBC was providing the wrong number of stable calls to the SLB.

Steps to Replicate: 

1. Bring up the SLB-ISBC setup.
2. Run a call load.
3. Check in the SLB about current utilization of the I-SBC.

The code is modified to provide the current number of stable calls in the utilization message to the SLB.

Workaround: None.

PortFix SBX-112487: The LI connection to media server was not connected on a switchover.

Impact: When using the PCSILI (P-com-session-info flavor of LI) in an N:1 M-SBC setup and running the SBC release 8.2 or later if there is a switchover, the connection to the LI server might not be re-established.

Root Cause: The standby instance was trying to maintain information about the operational status of the interface used for LI processing. But this was only happening correctly for the first instance. The status of the other instances was being misinterpreted and lost. So during the transition to active, the LI code thought the interface was not available and did not try to establish the connection to the LI server.

Steps to Replicate: In an N:1 setup perform switchovers from each active instance to the standby and check that the connection to the LI server is restored.

The code is modified to collect the status of the interface after the instance transitions from standby to active so that the accurate interface information is available for LI processing.

Workaround: Bounce the interface e.g. ifup/ifdown that is used for LI connection would help to recover the connection.

PortFix SBX-112493: The SBC is not decrypting the IPsec packet when a large PDU was sent over IPv6 from Strongswan.

Impact: The SBC is not decrypting the IPsec packet when a large PDU was sent from Strongswan IPSec endpoint.

Root Cause: The SBC SWe NP has a issue in last fragment data length processing w.r.t. This StrongSwan fragmented first and encrypted the next combination IPSec packets handling, which caused an ESP trailer offset being incorrect and resulted in call failures.

Steps to Replicate: Make large SIP PDU calls with the StrongSwan IPSec endpoint.

The code is modified to work for all combinations.

Workaround: Racoon/Navtel/SBC IPsec endpoints can be used if possible as a workaround.

PortFix SBX-112953: The SRTP license counter did not incremented when the called side omits an SIP 18x response.

Impact: The license count for the SRTP license was not updated on a call from SRTP to RTP, if no 18x response message is received.

Root Cause: The SRTP license usage is not updated at the ingress when response to an INVITE is only 200 OK message. Therefore, if the egress is not using a SRTP, the call does not consume a license.

Steps to Replicate: Make an SIP-SIP call where the ingress leg uses SRTP and egress leg uses RTP.
The called side does not sent any 18x response.

The code is modified to correctly update the license count at ingress, even when no 18x is sent.

Workaround: None.

SBX-113610 | SBX-114155

PortFix SBX-113610: Deleting of an adProfile entry caused the PES Process to coredump.

Impact: The PES Process dumps core when an AD profile entry is deleted.

Root Cause: While synchronizing the data from remote domain controller, the PES fetches the AD profile once from the cache and uses the object for various decision making. So, if the ad profile is deleted, the object reference becomes NULL, and it dumps core.

Steps to Replicate:  To create an AD profile, create the dependent profiles.

1. Create an AD Attribute Profile.
   set profiles adAttributeMapProfile DEFAULT_AD_ATTRIBUTE_PROFILE adAttributeList adAttribute1 adAttributeName cn
   set profiles adAttributeMapProfile DEFAULT_AD_ATTRIBUTE_PROFILE adAttributeList adAttribute2 adAttributeName displayName
   commit
2. Create a domain controller profile
   set global servers domainController dc0 userName DcUsername password DcPassword primaryAddress DcIpAddress searchScope base=engineering,dc=some,dc=com ldapQueryCriteria cn=*
   commit
3. Create an AD profile:
   set profiles adProfile DEFAULT_AD_PROFILE delayedSync 2021-09-29T14:00:00 syncInterval 1440 sync enable adServerList 1 dcServer dc0
   commit

After 1-2 minutes of creating the AD profile, delete it.

The code is modified to prevent deleting the AD profile. If you require, for some reason, to stop the synchronization, disable the sync option on the AD profile.

Workaround: If the AD profile is not required, do not  delete the AD Profile but disable the sync option on it. This will stop all the future auto sync operations.

portFix SBX-113766: The SCM process crash was observed when trying to show the ARS blocklisted entries.

Impact: When too many peers are in ARS blocklisted table(more than 90 entries) and the "show status addressContext default zone <ZONE_NAME> sipArsStatus" command was issued, the SCM process crashed.

Root Cause: The code was incorrectly indexing to a negative location in an array that led to an invalid memory access and a coredump.

Steps to Replicate: Block list multiple peers and then run the status command. This was only seen once in lab testing.

The code is modified to avoid accessing the negative array location to avoid the coredump.

Workaround: None.

PortFix SBX-113839: The customer setup is going for continuous reboot after upgrading (Stack Delete and Create) from 8.2.2R7 and 8.2.2R8 to 10.1.

Impact: The SWe_NP process exits during DPDK ACL tries to build operation causing the SBC to go for reboots.

Root Cause: postgres process is eating up significant number of huge pages from the quota allocated for SWe_NP. As a result, SWe_NP runs short of huge pages for DPDK ACL trie build operations and exits.

Steps to Replicate: 

1. Bring up a SBC instance with 10.1 release.
2. Apply the same NTT customer configuration. (primarily having same set of ACLs)
3. Verify that the SBC is stable after the configuration.

The code is modified to restrict the postgres process from consuming huge pages.

Workaround: Increasing number of huge pages could be a possible work around.

An OA fsm timeout after a call transfer with the e2eAck enabled (ACK not sent by the SBC).

Impact: After a call transfer from legB to legC, the e2e re-INVITE and ACK is not working.

Root Cause: LegC incorrectly disables the  e2e ACK, while legA still enables the e2eACK.

Steps to Replicate: This is specific to customer configurations.

1. A call B, B refer to C, with tone places and various codec setup.
2. Once LegC received 183 with the SDP, it triggers the SBC to send an UPDATE to legC.
3. LegC answers the 200OK INVITE, and the 200OK Updates.
4. LegC sends keepalive to trigger a re-INVITE to legA.
5. The SBC is unable to send ACK to the legA.

The code is modified to address the issue.

Workaround: Disable the e2eAck.

PortFix SBX-101239: The congestion seen in a SBC with no traffic.

Impact: The SBC 5400 reporting congestion even when no calls active.

Root Cause: On the 5400 server, the EMA has allocated 4 CPUs for java and there are times when these can all run hot even when no calls on the system. The minimum number of hot CPUs to trigger congestion on the 5400 was set to 4 so it could report without any calls.

Steps to Replicate: Enable SIP ladder diagram and CDR viewer on check for congestion reports on an idle system.

The code is modified to avoid congestion when no calls. This is similar to the other 5xxx models where the hot CPUs is one more than the number of java CPUs in EMA.

Workaround: Disable SIP ladder diagram and CDR viewer services to reduce the java CPU usage.

The SBC returned a 500 Internal Server Error and many calls failed.

Impact: Observed a memory leak when the ACK sending failed towards SIP recording server due to a DNS resolution failure.

Root Cause: Once a 200 OK received from SIP Rec server for initial INVITE/re-INVITE, current design assumes that INVITE/re-INVITE transaction is success and it is designed to free call control block only on completing BYE transactions.

But if ACK sending failed due to DNS resolution, then the SBC does not trigger BYE during call cleanup and in this case call control block is not getting freed, causing memory leak.

Steps to Replicate: 

1. Basic call with SIP Rec feature enabled.
2. Simulate SIP Rec server to send FQDN in 200 OK (200 OK of INVITE) and make sure DNS resolutions fails and this cause memory leak.

The code is modified to hold the call control block only when the SBC triggers BYE towards SIP Rec server and to clear call control bock when cleanup timer expires, in case BYE transaction fails to free call control block.

Workaround: None.

The coreProcesses.py needs to collect the pmap.

Impact: Starting with the 8.0 release, the Ribbon core dump analyzer also requires a pmap of a process when analyzing a core dump produced using the gdb gcore command.

Root Cause: With the 8.0 release, the stretch version of the Debian release was used. With this release, the information needed by the core dump analyzer was no longer provided in the core dump created by gcore.

Steps to Replicate: 

1. Use the a utility to consume 88% of the memory on the SBC.
2. Verify that each core file in the /var/log/sonus/oom has a corresponding pmap file.

The code is modified to collect a pmap of the process as well as the core file

Workaround: The core files must be loaded on an SBC running the release corresponding to the core file.

PortFix SBX-105367: During a sRTP and fax call, the sRTP context is removed for G711 fax pass-thru.

Impact: The sRTP context is removed on the leg that has a G711 fax when there is a t38 to g711 fax call.

Root Cause: The sRTP context is not updated for G711-fax.

Steps to Replicate: 

1. Run an A(SRTP)-B(RTP) CALL.
2. Run a leg A is G711 fax and B is T38. B sends a re-INVITE with t38 fax.
3. Check the re-INVITE that goes out with SRTP to endpoint A.
4. A party sends 200 ok with SRTP.
5. End the call.

The code is modified so the the SRTP context is updated only if the calleg has G711 fax.

Workaround: None.

PortFix SBX-109006: The DSP DHC had a Failure and failed to coredump.

Impact: The FPGA based DSP Health Check (DHC) fails and as a result, no DSP coredump is collected. 

Root Cause: The root cause for DHC failure is not known. However, subsequent coredumps failed due to a lack of reception of the DSP BOOTP packets, which is a hard failure. It is speculated that both issues are related.

Steps to Replicate: Instrument the code to replicate the issue.

Reboot the node instead of running an application restart to address the issue.

Note: This is not a fix for the original issue, but just a proposed workaround to prevent or reduce further failures.

Workaround: None.

PortFix SBX-112905: The SBC was answering an on-hold call with sendrecv then re-inviting.

Impact: When there is an invcoming INVITE on-hold, the SBC responds with 18x offhold and later a re-INVITE onhold.

Root Cause: There was a logical error that converts from inactive to sendrecv when the first 18x is send out. This issue was introduced from a previous fix. 

Steps to Replicate: 

1. On the ingress, configure the Minimize flag, and uncheck the relay data path mode.
2. Incoming INVITE onhold, egress response 18x.
3. The SBC sends a 18x to the ingress with sendrecv.

The original fixed of SBX-105961 should applied for subsequent 18x only.

Workaround: Enable relay data path mode

The SBC discards incoming STUN packets after a second call hold - call switched from a DM call to a pass-through call when going on-hold.

Impact: When running a call with ICE changes from direct media to running a call through a passthru due to 200 OK response, the ICE learning does not get enabled on the call.

Root Cause: The code was not re-enabling the ICE learning when 200 OK caused a change from direct media to passthru.

Steps to Replicate: 

1. Establish a call with ICE on ingress and xdmi on egress, such that call is established as direct media.
2. Send a re-INVITE from ingress to put call on hold, when re-INVITE is received at egress, respond with 200 Ok without xdmi and complete the re-INVITE related signaling.
3. Send stuns to ingress media to complete ICE learning and verify call state changes to established and ICE learning shows as completed.
4. Send a re-INVITE from ingress to take call off hold, when re-invite is received at egress, respond with 200 Ok without xdmi and complete the re-INVITE related signaling.
5. Verify media can be exchanged between ingress and egress.

The code is modified to re-enable the ICE learning when changing from direct media to passthru to allow receipt and processing of stun messages to complete ICE learning.

Workaround: None.

PortFix SBX-101409: The T140 call on a one-way stream had zero media port (NAPT media).

Impact: The call ends up in one-way audio after the called party puts the call on hold and off hold twice.

Root Cause: As a result of running a call-modify multiple times, the RTCP NAPT learning completes before the RTP NAPT learning.

This results in RTCP Remote Address being updated, which has remote RTCP Port.

Due to incorrect code in RTCP modify flow, the remote RTCP port is assigned to RTP port. This results in one-way media.

Steps to Replicate: 

1. Run basic SIP to SIP call with NAPT and RTCP enabled.
2. Hold/Unhold the call a multiple times to check for proper two-way audio.

The code is modified to set the correct RTP Port as part of RTCP modify flow.

Workaround: Since the issue is caused to RTCP NAPT learning completed before RTP during multiple hold/Unhold scenario. Use the following workaround: 

1. Disable RTCP OR
2. Disable NAPT.

The SBC discards incoming STUN packets after a second call hold and the call switched from a DM call to a passthru call when going on-hold.

Impact: When a call with ICE changes from direct media to passthru due to 200 OK response, ICE learning does not get enabled on the call.

Root Cause: The code was not re enabling the ICE learning when 200 OK caused a change from direct media to passthru.

Steps to Replicate: 

1. Establish call with ICE on ingress and xdmi on egress, such that call is established as direct media.
2. Send a re-INVITE from ingress to put call on hold, when re-INVITE is received at egress, respond with 200 Ok without xdmi and complete the re-INVITE related signaling.
3. Send stuns to ingress media to complete ICE learning and verify call state changes to established and ICE learning shows as completed.
4. Send a re-INVITE from the ingress to take call off hold, when an re-INVITE is received at egress, respond with 200 Ok without xdmi and complete the re-invite related signaling.
5. Verify that the media can be exchanged between ingress and egress.

The code is modified to re-enable the ICE learning when changing from direct media to passthru to allow receipt and processing of STUN messages to complete ICE learning,

Workaround: None.

PortFix SBX-112547: The SBC routes call to 2nd DNS record if 503 is received after 18x.

Impact: The SBC routes an INVITE to next DNS record if 503 is received after 18x.

Also, even when the dnsCrankback flag is disabled, on getting 503/INVITE timeout case, the SBC reroutes an INVITE to next DNS record.

Root Cause: Due to a design defect, the SBC tries the next DNS record even if an error response is received after an 18x.

Also, retrying the next DNS record during a 503/timeout when dnsCrankback flag is disabled is legacy behavior. This behaviour is changed to retry the next DNS record only when the dnsCrankback flag is enabled.

Steps to Replicate: 

1. Configure the DNS server with two routes for the FQDN route.
2. Make an A-to-B call.
3. The SBC sends an INVITE to first DNS record of and B sends 18x and the 503.

The code is modified to not apply the DNS crankback procedure if an error response is received after a 18x. Additionally, the default behavior of handling timeout/503 when the dnsCrankback is disabled is updated as part of this fix.

With this fix, the SBC retries for the next DNS record only when dnsCrankback is enabled to ensure the error responses match the reasons configured in the crankback profile.

Workaround: None.

Portfix SBX-104733: SCM Process core dump during healthcheck.

Impact: The SCM Process core dumped when too many set operations executed in a single commit.

Root Cause: The Call Processing tasks takes longer than 10 seconds when subscribing to the changes made to SIP Trunk Group.

Steps to Replicate: Configure 12 trunk groups. Modify 11 trunk groups with a single commit command and the CLI will deliver the following error message:
Aborted: 'addressContext default zone ZONE_IAD sipTrunkGroup': Too many set operations between commits. Revert the session and retry(max set operations 10).

Again modify 10 TG in single commit: 
O/P: commit complete

The code is modified per commit is changed to 10 from earlier value of 50.

Workaround: Ribbon recommends performing a commit after every CLI transaction.

PortFix SBX-111751: The SCM Process is coredumping.

Impact: The SBC is relaying multiple reason header instances in the amount of the quantity squared.

Root Cause: Logical error when multiple instances of a reason header on the same line. The SBC is relaying in the amount of quantity squared. 

Steps to Replicate: 

1. Configure the transparency Reason header.
2. Make a SIP-SIP call, Ingress sends BYE with multiples Reason header instances on the same line. The SBC sends instances in an amount of the quantity squared to the Reason header on Egress.

Note: If the ingress sends a large number of instances on the same line.

• For old (pre 8xx), the SBC may core.
• For 8xx and above, the SBC may be unable to send BYE on Egress.

The code is modified to address the issue.

Workaround: Disable the transparency of a reason header.

PortFix SBX-104619: The FM process coredumped. 

Impact: The FM Process crashed and wrote coredump.

Root Cause: The FM Process tried to read the /proc/meminfo file, which should always exist, but it got a file not found error.

Steps to Replicate: It is not known how to reproduce this as this should never happen, so defensive code added to prevent null read/write.

The code is modified so when we cannot read the /proc/meminfo file, we return the last good value read instead of a NULL to prevent the crash.

Workaround: None.

Portfix SBX-109177: The SbcSftp throws an error as "Failed to add ACL".

Impact: The sbcsftp application does not remove the ACL created correctly.

Root Cause: The permissions to delete the ACL gets lost while lowering permissions to ensure the sbcsftp can only access the current user's files.

Steps to Replicate: 

1. Run SbcSftp as linuxadmin. Verify no 'Failed setting permissions' error messages appear.
2. Verify Cleanup Script:
   1. Run SbcSftp but kill the the process before it completes.
   2. As root, get the name of the ACL.
   3. Wait 15 minutes.
   4. Verify ACL removed.

The code is modified so that we can raise the permissions again once SFTP is complete.

Workaround: None.

PortFix SBX-109013: Increased the healthcheck interval.

Impact: Core dumps are occasionally seen in the field due to the health check timer expiring when the process gets stuck.

Root Cause: In SWe environments, disk/CPU timing issues can lead to the process slowing down and hitting these health checks.

Steps to Replicate: This issue was only reproduced using debug code.

The code is modified to be more forgiving to cope with short spikes. The health check ping interval is now 2 seconds and needs to have 15 consecutive non responses in order for the process to be declared deadlocked and a coredump initiated to recover.

Workaround: None.

PortFix SBX-104866: ENM Process is leaking memory.

Impact: The memory usage for the ENM Process increases when CDR file transfer failures are present.

Root Cause: Under certain file transfer scenarios, the third party library libssh2 used by the ENM process leaks memory.

Steps to Replicate: This problem was reproduced by creating a custom version of libssh2 that randomly dropped packets received from the remote sftp server.

The code is modified to keep track of the memory allocated by libssh2, and to free up any memory still in use after the file transfer session is closed.

Workaround: N/A

PortFix SBX-111720: There was an SCM Process core in the SMM area.

Impact: The SBC may coredump when using the SMM and regex to modify the fieldvalue of request-line.

Root Cause: The SBC was accessing a corrupted pointer, which caused the issue.

Steps to Replicate: Run a SMM using regex to modify fieldvalue of request-line.

Note: This may not be reproducible due to the nature of the access pointer.

The code is modified to address the issue.

Workaround: Change the fieldValue to headerValue. Or change the rule not to use regex.

PortFix SBX-108356: The SBC has various runtime errors found in np.log.

Impact: Internal ASAN builds report runtime errors on SWe platform related to left shift of signed integers.

Root Cause: Appropriate integer casts were missing in code, which caused ASAN runtime warnings related to bit shift.

Steps to Replicate: Bring up ASAN build on SWe platform and observe if np.log contains ASAN runtime warnings related to bit shifts.

The code is modified to fix the runtime warnings in ASAN builds.

Workaround: No workaround.

PortFix SBX-106197: The PKT port manualSwitch triggered twice.

Impact: Executing port switchover command from CLI triggers port switchover twice.

Root Cause: Link detection sub system on standby node subscribes to port switchover action command with CDB (Confd) twice:

1. When the node comes up as standby.
2. When it initiates switchover to become active.
   
   When standby node becomes active, executing port switchover command from CLI results in notifying link detection sub system twice due to duplicate subscription. This results in initiating port switchover twice.

Steps to Replicate: 

1. Bring up nodes with port redundancy in HA mode.
2. Initiate CE switchover to make sure standby node becomes active.
3. Execute port switchover command on new active node and validate the behavior.

The code is modified to not subscribe for action command when the node is in standby mode and subscribe only upon switchover when it transitions from standby to active.

Workaround: None.

PortFix SBX-106788: TOD Routing was broken for non-ALL timeRangeProfiles.

Impact: Time of Day Routing is broken for any configured time range profile other than the default ALL profile.

Root Cause: The time of the day values are stored in the DB as hexadecimal octets and A to F digits are stored as lower case.

While matching the configured data, the stored values are compared against upper case A to F digits, thus the result of this match always failed.

Steps to Replicate: Set timeRangeProfile other than ALL to test.
set global callRouting route trunkGroup INTERNAL_IPTG99 VSBCSYSTEM standard Sonus_NULL 1 all all TEST0001 none Sonus_NULL routingLabel TO_EXTERNAL_TG99

Note: The TRP TEST0001 is defined as below, and is set to match all days and all times:

set profiles callRouting timeRangeProfile TEST0001 entry 7 timeZone psxLocal

set profiles callRouting timeRangeProfile TEST0001 entry 7 dayMatching dayOfWeek monday,tuesday,wednesday,thursday,friday,saturday,sunday

set profiles callRouting timeRangeProfile TEST0001 entry 7 dayMatching holidays disable

set profiles callRouting timeRangeProfile TEST0001 entry 7 dayMatching specialDays range none

set profiles callRouting timeRangeProfile TEST0001 entry 7 timeMatching range all

The code is modified  to check for both lower case and upper case hexadecimal digits.

Workaround: No workaround.

Portfix SBX-111310: Standby OAM is rebooting after orchestration.

Impact: The Standby OAM is restarting after a launch.

Root Cause: The Standby OAM is sending the serf event for 'startingStandby' with an older timestamp (which is fetched even before standby OAM is ready to come up) that results in discarding of the serf event by the active OAM node as its prior to member-join event for standby OAM node.

Steps to Replicate: Bring up both the active and standby OAM at the same time and then while standby OAM is coming up, disconnect and connect the HA port on active OAM to trigger member-failed and member-join events.

While sending the 'startingStandby' serf event, get the current timestamp so that its not discarded by the active OAM node.

Workaround: Restart the standby OAM application so that the startingStandby event is generated again with a current timestamp.

Portfix SBX-110179: The LeakSanitizer: detected memory leaks.

Impact: A memory leak occurs when a logical management interface is added or modified.

Root Cause: A confd cursor element was not closed when exiting the routine that validates the logical management interface being added or changed and this resulted in a memory leak.

Steps to Replicate: Make changes to the logical management interfaces and check for memory increasing in the Cpx process.

The code is modified to ensure the associated memory is freed to avoid the leak.

Workaround: None.

Portfix SBX-106543: The SBC experiences a core dump while running UAS Notify Request.

Impact: The SBC coredumps while processing the UAS Notify Request within the XML body.

Root Cause: In API SipsCheckForAnyBody(), the loop where the string pointer pch is incremented each time was being accessed before validating for an out of bounds condition that caused the crash.

Steps to Replicate: Send a NOTIFY with XML body from UAC towards the SBC.

The code is modified to add a out of bounds check before accessing the pointer value.

Workaround: None.

Portfix SBX-105900: Resize log volume on every boot.

Impact: The log volume is not being resized on every boot.

Root Cause: Currently, we build the filesystem on cinder volume only on the first boot.

Steps to Replicate: 

1. Create and attached a cinder volume of 50 GB.
2. Bring down the SBC, resized cinder volume to 100 GB.
3. Launch the SBC with the cinder volume attached.

Check if the cinder volume is resized to 100 GB.

Check if file system is already built. If the file system is not built, resize the volume to address the issue.

Workaround: None.

PortFix SBX-105160: There was a CHM process coredump on standby OAM after reboot.

Impact: The SBC application continues to come up even when asp.py zap (zapAsp() call in sbxCleanup.sh) is invoked from sbxCleanup.sh script. When we try to access ceNum in one of the places, since it is not properly set due to failure in sbxCleanup.sh, it results in CHM coredump

Root Cause: The SBC application continues to come up even when asp zap is called.

Steps to Replicate: Fail sbxcleanup in one of the places and ensure the SBC in not being brought up.

The code is modified to avoid the SBC bring up if any errors are reported in sbxCleanup.

Workaround: The issue is fixed, no workaround is required.

Portfix SBX-105856: The Wrong Version in URL after a Restore to an older version.

Impact: The wrong version in URL after a Restore to an older version.

Root Cause: URL is picked from CDB from path "/system/objectStoreParameters/url" that is independent of software version of revision.

Steps to Replicate: 

1. Spawn OAM act, standby and one SSBC with V10.00.00A008.
2. Create multiple revisions (e.g., rev1, 2, 3, 4, 5)
3. Upgrade cluster to V10.00.00A009.
   
   [root@VSBCSYSTEM-2-vsbc2 172.12.34.567 evlog]# cat /mnt/gfsvol1/config-versions.txt
   5,V10.00.00A008,oam_config_20210510T042937.tar.gz,0,2021-05-10T04:29:37,,downloaded_from_ems
   6,V10.00.00A009,oam_config_20210510T043100.tar.gz,1,2021-05-10T04:31:00,REBOOT_REQ,auto_save_after_software_change
   [root@VSBCSYSTEM-2-vsbc2 172.12.34.567 evlog]#
4. Manually reboot and replace the SM Process with a fix.
5. Restore revision 3.
   
   7,V10.00.00A008,oam_config_20210510T045034.tar.gz,6,2021-05-10T04:50:34,RESTORE,restored_from_3
   8,V10.00.00A009,oam_config_20210510T045311.tar.gz,6,2021-05-10T04:53:11,REBOOT_REQ,auto_save_after_software_change
   
   Observation: Revision 7 uploaded with correct URL.
6. Remove revision 7 from /mnt/gfsvol1 and restore revision 7.
   Revision successful and newly created revision uploaded on EMS with correct URL
   11,V10.00.00A008,oam_config_20210510T055545.tar.gz,20,2021-05-10T05:55:45,RESTORE,restored_from_9
   12,V10.00.00A009,oam_config_20210510T055745.tar.gz,20,2021-05-10T05:57:45,REBOOT_REQ,auto_save_after_software_change

The code is modified to update the URL with software version of revision being restored.

Workaround: None.

PortFix SBX-109493: Forwarding info was redirecting info for a customer.

Impact: When testing with JJ90.30 call flows and sending in the following parameters, the RDI (redirection information) parameter created from the history header information had the redirecting indicator field set to “call diverted, all redirection information presentation restricted” even though there was no privacy parameter information in the history header.

Root Cause: The SBC interworking code was following the 3GPP 29.163 specification table 7.4.6.3.2.3 that allows the standard SIP privacy header information to be used when setting the RDI parameter information. Due to the setting RDI parameter information "id", it resulted in the redirecting indicator parameter value of “call diverted. All redirection information presentation restricted” instead of it being set to "call diverted".

Steps to Replicate: Send an INVITE to the SBC, with parameters similar to those in the impact statement and check in the policy request that the RDI redirecting indicator field is set to "call diverted". The SIP trunk group variant needs to be set to TTC and the signaling acceptHistoryInfo control needs to be set to enabled.

The code is modified to not consider the SIP Privacy header value when determining the RDI parameter redirecting indicator field value if the SIP trunk group variant is set to TTC.

Workaround: None.

Portfix SBX-109966: The SBC incorrectly accepts an SDP offer in an ACK

Impact: The SBC is designed to ignore an SDP offer in an ACK, but is not doing so.

Root Cause: In an early media call, the SIP stack was accepting the SDP in ACK as an offer. This should never happen.

Steps to Replicate: 

1. In an early media call, add SDP in ACK.
2. Send a re-INVITE with SDP from the same side.
3. SIPS stack should ignore the SDP in an ACK and accept the SDP from re-INVITE as an offer.

The code is modified to not forward the SDP to the application.

Workaround: None.

Portfix SBX-108058: The SBC CpxAppProc memory leak.

Impact: During the SBC startup processing there is a small memory leak.

Root Cause: During the startup processing, the SBC is allocating memory while reading configuration information and it is not being freed up correctly at the end of the provisioning steps.

Steps to Replicate: This issue was only observed while running with ASAN images in the engineering lab as the amount of memory leaked is small and cannot be checked.

The code is modified to correctly free the memory allocated while processing the configuration data.

Workaround: None.

Portfix SBX-109681: Low MOS score for UNENCRYPTED_SRTP calls

Impact: In SRTP for unencrypted, Authenticated case, SRTP packets were being discard at NP due to authentication key mismatch.

Root Cause: In SRTP for unencrypted, authenticated combinations key size is required in NP to derive the session authentication keys. Since the cipher key size was not being pass to NP, session authentication key was wrongly calculated.

Steps to Replicate: On the receipt of an SDP offer with crypto attributes, if 'enable SRTP' is Configured in packet service profile, a common crypto suite is found in the crypto attribute received in the offer and in the packet service profile, and session parameters if included are allowed, the offer will be accepted. In the answer, the SBC includes the same tag used in the offer.

Test Setup on the SBC:

1. Endpoint1 Configured with SRTP profile( SRTP/SHA-1-80). Disable all Session Parameters in packet service profile. Allow Fallback enabled.
2. Endpoint2 Configured with SRTP profile(SRTP/SHA-1-80).Enable Session Parameters UNENCRYPTED_SRTP in packet service profile. Allow Fallback enabled.

Procedure:

Endpoint1 sends an offer SDP with crypto attribute SRTP/SHA-1-80 and with Session Parameters UNENCRYPTED_SRTP
Endpoint2 replies with crypto attribute SRTP/SHA-1-80 and with Session Parameters UNENCRYPTED_SRTP

The code is modified so the SBC application is passing cipher key size also to NP to calculate session authentication keys.

Workaround: Workaround is to not send unencrypted SRTP. Use cases with Encrypted SRTP and authentication no issue is seen.

Portfix SBX-108370: The top2 on the SBC core takes lot of CPU cycles.

Impact: The SBC performance monitoring tools like top2 at times take 20% of a CPU core there by reducing total available CPU resources for management activities on SBC.

Root Cause: Introduced nice values to sbxPerf commands but still top2 taking around 20-30% at times. As a result, need to check ways to optimize it to take less CPU using /root/.top2rc.

Steps to Replicate: Install fix build and by using top command make sure CPU utilization of top2 should not be much CPU utilized.

The code is modified to take less CPU utilization of the SBC performance monitoring tools like top2.

Workaround: None.

PortFix SBX-105890: On a disk failure, the correct failover is not triggered.

Impact: On an SBC that was running 6.2 code, the disks on the SBC got into a bad state and would not process read or write operations. Automatic attempts to reboot the SBC from the application code failed and manual intervention was required to recover.

Root Cause: Later software releases already had better handling for this sort of issue but the code was printing multiple logs as part of the automatic reboot process and it was suspected that these could have gotten hung and the system could not get to the reboot command.

Steps to Replicate: The issue was not reproducible.

The code is modified to remove the logs to avoid the potential or not getting to actual reboot command in the code.

Workaround: The SBC needs to be manually power cycled to recover.

PortFix SBX-111358: Customer ECGI to CA Mapping Enhancement.

Impact: The SMM Switch operation is not working after an SBC reboot.

Root Cause: The switchIndex is not being set properly during a config restore.

Steps to Replicate: 

1. Configure SMM Rule consisting of switch operation.
2. Run the Call.
3. Perform an SBC reboot.
4. Run the call (SMM will not be executed).

The code is modified to set correct the SwitchIndex during a config restore.

Workaround: After a reboot, we can delete the SMM profile and configure again.

PortFix SBX-108369: The FIPS mode was broken on the AWS SBC.

Impact: After enabling the FIPS mode on the AWS SBC or OpenStack, the SBC functions such as 'certificate import' do not work.

Root Cause: While enabling FIPS mode, the FIPS flag is set in sbx.conf. But on Openstack/AWS deployments, the sbx.conf gets reset on the reboot causing a reset of the FIPS flag.

Steps to Verify Fix: 

1. Enable fipsMode from CLI.
2. After a reboot, check /opt/sonus/conf/sbx.conf or try a certificate import from the CLI.

The fipsMode in sbx.conf is set to 'enabled'.

The code is modified to retain FIPS flag in sbx.conf on reboot.

Workaround: After enabling the FIPS mode, set fipsMode=enabled in /opt/sonus/conf/sbx.conf.reconfigHa.orig and reboot the SBC.

After a reboot check, /opt/sonus/conf/sbx.conf

PortFix SBX-110948: Load configuration overwrites the local processor index values.

Impact: When we perform a load config operation on a 1:1 HA pair (SWe/Cloud), it ends up overwriting the local processor index estimates with the ones that are present in the config dump.

This leads to two issues:

1. An incorrect index estimates being populated in the DB.
2. There are discrepancy in estimations between active and standby instances.

Root Cause: The load config operation results in overriding the local processor index estimates with the ones that are present in the config dump. This results in standby consuming incorrect indices present in the DB thereby causing the discrepancy in estimates of the active and standby instances.

The DB should always reflect the estimated processor indices calculated by the active instance in 1:1 HA pair.

Steps to Replicate: On 1:1 HA SWe/Cloud instances, perform the following steps:

1. Run the following command as root user on the active and standby instances. The following command should give same output on the active and standby instances.
   cat /opt/sonus/conf/swe/capacityEstimates/.index.txt
2. Perform the load config operation.
3. Run the following command as root user on the active and standby instances. The following command will give different output on the active and standby instances.
   cat /opt/sonus/conf/swe/capacityEstimates/.index.txt
   
   With the fix, this issue will not be observed.

The code is modified to ensure that the DB reflects the estimated processor indices calculated by the active instance in 1:1 HA pair.

If the processor indices values stored in the DB does not match with the indices calculated by the standby, then the standby goes for a reboot. From the next boot on-wards, the standby uses the indices stored in the DB for the session estimations.

The previously mentioned procedure ensures that the indices and sessions estimates are same on the active and standby instances.

Workaround: Run the following commands as root user on the active and standby instances.

1. The rm -f /opt/sonus/conf/swe/capacityEstimates/.indexMarker
2. Run a reboot

PortFix SBX-111163: Export/import of the syslog configuration fails.

Impact: The user-config-import command fails if the customer has configured the SBC to send the Linux level logs to a remote syslog server through the platformRsyslog configuration.

Root Cause: The child configuration objects under the platformRsyslog configuration were not being applied in the correct order during the import, which led to the configuration validation logic thinking there was no syslog server configuration and failing.

Steps to Replicate: Apply syslog configuration under the platformRsyslog and check that it can be exported and imported without errors.

The code is modified to correctly define the configuration order for platformRsyslog configuration so there are no errors during the import.

Workaround: If you edit the syslogState line in the xml file and change it from enabled to disabled, the import will complete correctly. Then manually apply the CLI command to enable the syslogState once the rest of the configuration is imported.

<platformRsyslog>
<syslogState>disabled</syslogState>
<linuxLogs>
<platformAuditLog>enabled</platformAuditLog>
<consoleLog>enabled</consoleLog>
<sftpLog>enabled</sftpLog>
<kernLog>enabled</kernLog>
<userLog>enabled</userLog>
<daemonLog>enabled</daemonLog>
<authLog>enabled</authLog>
<syslogLog>enabled</syslogLog>
<ntpLog>enabled</ntpLog>
<cronLog>enabled</cronLog>
</linuxLogs>
<servers>
<serverNo>server1</serverNo>
<remoteHost>2607:f160:10:4043:ce:ff0:0:35</remoteHost>
<protocolType>udp</protocolType>
</servers>
</platformRsyslog>

PortFix SBX-109298: SBC: The DSP fails to modify ptime.

Impact: When the SBC receives a re-INVITE with a change in the ptime for EVS, the DSP Modify command fails.

Root Cause: Handling of DSP Modify Command for ptime change for EVS was not present.

Steps to Replicate: Run a EVS to G711 call, send a re-invite on the EVS leg with change in ptime. The change in ptime should be put into effect through a DSP Modify.

The code is modified to support for ptime change for EVS is added. Allowable ptime changes include 20, 40, 60, 80 and 100ms.

Workaround: None.

PortFix SBX-109298: Observed SIPFE MAJOR logs on the n1-standard-4 SBC_HA_HFE_SPLIT instance.

Impact: The log was printed as major, when stale calls are present and whenever we start cleaning the stale calls, then the log is seen.

Root Cause: This log is expected when clearing any stale calls.

Steps to Replicate: Run call load and if any stale calls are seen, then this log issue is seen (only when the Minor logging is enabled).

The code is modified to address the issue.

Workaround: None.

Portfix SBX-106589: The SBC SWe does not forward the SIP Info and drops the call.

Impact: After receiving 32763 SIP indialog INFO messages, the SBC sends a 500 error for further INFO messages instead of forwarding the message.

Root Cause: The SBC maintains the callHandle structure with refcount, which will be incremented for every reference and decremented once reference is released. But in this case, the refcount is not decremented properly, for every indialog msg (INFO) request received, the count kept increased.

Steps to Replicate: 

1. Establish a call.
2. Send indialog INFO messages more than 32763.

The code is modified to address the issue.

Workaround: No workaround.

PortFix SBX-109407: MicroFlows are failing in the REGISTER Performance of 2400 RPS (256000 REGISTRATIONS) with an error code 0xf9.

Impact: Unable to support the 256k concurrent subscriber registrations in Default memory profile for the SBC SWe.

Root Cause: Existing logic for determination of flow hash causes increased number of hash collisions leading to increased depth of hash buckets, leading buffer starvation.

Steps to Replicate: 

1. Spawn a SBC SWe instance (of default memory config profile).
2. Test the 256K registrations.

The code is modified to minimize hash collisions.

Workaround: There is no workaround for this.

PortFix SBX-105175: The SBC sends a re-INVITE while media is played on the ingress side in a GW-GW early media call.

Impact: In a GW-GW call scenario, while the media is played on the ingress Gateway, the egress SBC is sending a re-INVITE to the UAS.

Root Cause: Before the ingress GW completes its end to end activation, it received a MODIFY OFFER request from the egress GW due to the change in SDP received in 200 OK for lockdown INVITE. This caused the ingress GW to process modify offer first and then end to end activation later that triggered a re-INVITE towards UAS.

Steps to Replicate: 

1. Send INV from UAC to GW1.
2. GW2 sends INV to UAS.
3. UAS sends 180 SDP.
4. GW1 sends 180 SDP to UAC and starts to play tone.
5. UAS sends 200 OK with out SDP.
6. GW1 sends 200 OK to UAC.
7. GW2 sends ACK and triggers a lock down INVITE.
8. UAS sends 200 OK with change in SDP.

The code is modified so that while modify offer-answer is handled properly during end to end activation.

Workaround: None.

PortFix SBX-108066: The /opt/sonus/sbx/bin/opensslSelftest cmd failed in the SBC 7000.

Impact: When the FIPS mode is enabled on hwType SBC7000 (7k), services will fail to come up and the SBC becomes inaccessible.

Root Cause: After enabling the FIPS mode when FIPS selfTests run, opensslSelfTests fails due to using a deprecated SSL engine on SBC 7000.

Steps to Replicate: 

1. Enable FIPS mode from CLI.
2. Ensure the SBC services are accessible after reboot.

The code is modified to remove the deprecated SSL engine from opensslSelfTest and this allows FIPS mode to function as expected.

Workaround: None.

PortFix SBX-105073: The *XrmAsyncCmdErrHdlr: ERROR NpMediaIntf cmd 2c gcid.

Impact: The set grace command returned an error code 0xF0 from the NP, which triggered unsolicited call cleanups.

Root Cause: The set grace command can only be performed on a disabled media flow in the NP. But, thet NP had received some set grace commands issued on an already enabled media flow. Unfortunately, we are unable to determine from just the core WHY XRM and NP became out of sync.

Steps to Replicate: The issue is not reproducible.

The code is modified to set after XRM has sent media flow enable command to NP and is cleared on media flow disable. The XRM checks this flag and only send set grace commands to NP when this flag is clear. If the flag is set, the XRM reads the  media control block in NP and log a MAJOR level debug message to help future debugging efforts.

Similarly introduced a set of new flags in the BRES structure for RTCP Gen enable/disable commands sent to NP. The BRM checks the new flags and only enables RTCP Gen in NP when the flag is clear. If the flag is set, BRM reads the rtcpGen control block in NP and log a MAJOR level debug message to help future debugging efforts.

Workaround: No workaround.

 PortFix SBX-110719: HA setup on ASAN is not coming up after call config - pathcheck and SMM

Impact: While testing with the following configuration the code could potentially read of the end of an internal memory block while printing a debug log. In the worst case scenario this could result in an SCM coredump.

set addressContext default zone <zonename> sipTrunkGroup <trunk group> signaling messageManipulation smmProfileExecution fixedOrder

Root Cause: The internal structure was not always getting initialized correctly prior to trying to print the contents and this led to the code reading pass the end of the memory block.

Steps to Replicate: Configure SMM with the fixedOrder configuration and make some calls. This issue was highlighted while running with ASAN images in the engineering lab.

The code is modified to correctly initialise the internal structure and avoid reading past the end of a memory block.

Workaround: Avoid using the "signaling messageManipulation smmProfileExecution fixedOrder" configuration if possible.

Portfix SBX-109167: The AddressSanitizer: SEGV on unknown address 0x000000000028.

Impact: During the pre-parsing the Messagebody, the SEGV on an unknown address is observed in SipsPreParseMsgBody(). This could result in an SCM coredump.

Root Cause: When the Content-Type is NULL the code was performing string comparisons to see if its an expected Content-Type and the code did not verify that the header pointer was not NULL before accessing it.

Steps to Replicate: 

1. Make an A to B call.
2. Send a re-INVITE with the MessageBody Content-Type as empty.

The code is modified to check the Content-Type string pointer is not NULL before accessing it.

Workaround: None.

Portfix SBX-107975: A Serf event processor is unable to restart because restart check marker file is not getting removed.

Impact: The Serf event processor is unable to restart because the restart check marker file is not getting removed.

Root Cause: The restart check marker was only being removed if serfeventProcessor starts successfully, so if it fails to start, any attempts to restart it would be prevented because the marker is not removed.

Steps to Replicate: 

1. Make serf event processor fail at some point.
2. It should try to restart up to 5 times if it keeps failing to come up.

The code is modified so that the user can attempt a restart.

Workaround: None.

Portfix SBX-107960: The AWS IPs remain assigned to the standby SBC. Unnecessary dual restarts.

Impact: If communication between active and standby is broken (over ha0 interface) both assume active roles (split brain). When this happens, the standby node that becomes active calls AWS APIs to move IP address to self. Once ha0 link is restored, the machine that becomes active does not call AWS API to move IP addresses, and this might cause issue when node that has IP does not come up as active. In this case, IPs are assigned on standby and another node becomes active.

Root Cause: Root cause of this issue is not calling AWS switchover API (move IPs) during split brain recovery.

Steps to Replicate: Perform a Split brain test and recovery of the SBC HA, and verify that API query is send by an Active SBC after recovery.

The code is modified to call AWS APIs to move IP to current active machine during split brain recovery path.

Workaround: Manually move all secondary IPs to current active machine to restore calls.

Portfix SBX-109418: The LeakSanitizer detected memory leaks Direct leak of 883820 byte(s) in 413 object(s).

Impact: The SBC was not freeing memory in one of the failure cases.

Root Cause: The SBC was not freeing memory in few cases where incoming INVITE handling fails.

Steps to Replicate: 

1. Send an INVITE with Replaces having call id, to in the Replaces header that does not match to any existing leg.
2. After the call ends, the ASAN should not show memory leak.

The code is modified to free up memory allocated, in all cases when the INVITE handling fails.

Workaround: None.

PortFix SBX-106629: While expecting the 200 OK, the SBC is sending 503 Service Unavailable.

Impact: In a late media convert call scenario with LRBT enabled, the SBC is sending 503 service unavailable when connect is received from egress EP.

Root Cause: The SBC was trying to initiate an UPDATE without checking whether it has received SDP in the PRACK from the ingress Peer that is resulting in the call failure.

Steps to Replicate: The LRBT is enabled on the ingress Trunk.

1. UAC sent LM INV to the SBC.
2. The SBC sent INV with PCMU, PCMA, G729 to UAS.
3. UAS sent 180 with PCMU to UAC.
4. UAC sent PRACK without SDP.
5. UAS sent 200 OK with same SDP as that of last 180.

The code is modified to check whether the SBC is received any SDP from the peer, so that, the call establishment occurs successfully.

Workaround: None.

PortFix SBX-111059: Memory leaks are observing on ASAN build when SMM is configured

Impact: While assigning SMM or flexible policy profiles to the zone configuration small amounts of memory leaked.

Root Cause: The code was allocating memory in order to read information from CDB as part of processing the assignment and the memory did not get freed up at the end of processing.

Steps to Replicate: Make SMM configuration changes at the zone level.

The code has been updated to correctly the free the memory.

Workaround: None.

Portfix SBX-110178: The PRS Process gave heap use after free on address on latest 10.0 build.

Impact: The PRS Process gave a "heap use after free on address" error while running the HA suite on ASAN build.

Root Cause: Interface number was being returned after typecasting the main structure to packet LIF structure, and not management LIF structure.

Steps to Replicate: Run SBX_504_HA suite on ASAN build.

The code is modified to correct the typecasting.

Workaround: None.

Portfix SBX-109439: SBC: An activeRevision failure when state for even type audit is set to on/off.

Impact: The configuration Playback on managed VMs fails on command.

set oam eventLog filterAdmin vsbc1 audit audit level major state off

Root Cause: The playback engine does not use user context.

Steps to Replicate: Run the command on the OAM.

set oam eventLog filterAdmin vsbc1 audit audit level major state off
commit

Perform a saveAndActivate.

The code is modified to address the issue.

Workaround: Reboot on managed VMs to get the configuration at startup.

Portfix SBX-111162:  The SBC fails to defragment/assemble fragmented IPSEC ESP packets received from the customer.

Impact: The SBC drops an in-coming 1500B IP fragments sent through an IPsec tunnel, where the ESP packet is also fragmented into approximately equal-sized IP fragment packets. This complex IP frag-IPsec-IP frag problem primarily affects large SIP INVITE packets in certain network designs.

Root Cause: This problem affects large packets sent through IPsec gateways that (1) do no reassemble received IP fragmented packets before sending them through the IPsec tunnel (IP fragments themselves are sent through the tunnel), and (2) fragment resulting ESP packets into approximately equal-sized packets instead of a 1500B and 72B fragments.

Steps to Replicate: Test should send SIP packets → 1500B to SBC over an IPsec tunnel through a device/devices that:

1. Fragments the SIP packet into 1500B + small IP fragments.
2. Sends the IP fragments through the IPsec tunnel.
3. Fragments (again) the larger ESP packet into approximately equal-sized IP fragment packets.

The code is modified to reassemble IP fragments up to 1580B divided any way into a single internal packet buffer, which the IPsec decryption code and subsequent IP frag reassembly code can handle.

Workaround: Two workarounds are possible:

1. Terminate the IPsec tunnel on a router in front of the SBC instead of directly on the SBC.
2. Do not send the IP fragments through an IPsec tunnel terminated on the SBC. Instead, reassemble IP packets before sending them through the tunnel towards the SBC, so that there is only one level of IP fragmentation (of the ESP packet itself).

PortFix SBX-111211: Get "violates foreign key constraint" error when assigning timeRangeProfile to a Route in EMA

Impact: Route creation fails when the name of time range profile contains numerals.

Root Cause: During the route creation, validation of time range profile was failing as it was containing numerals.

Steps to Replicate: 

1. Create a time range profile with a name containing number ex test_1
2. Create a route with test_1 as time range profile.
3. Route creation should be successful.

The code is modified to consider numerals as valid a character.

Workaround: Create a Route from the CLI.

Portfix SBX-109453: The SBC is closing the socket for DNS query over TCP frequently.

Impact: The SBC is closing the socket for DNS query over TCP frequently.

Root Cause: Whenever the TCP connection is closed, the FIN packet is sent towards the SBC from DNS server. But the application was not closing the connection by sending the FIN and Even after a connection is closed by DNS server, the application is still holding socket information. This details were used in future queries has cause TCP connection to reset.

Steps to Replicate: 

1. Have a v6 Cloud SBC with the below build
   SBC: V10.00.00-A006.
2. Configure the DNS server to send queries using TCP connection.
3. Make a call.
4. Wait for some time and run one more call here.

Actual behavior:

In step 3, the SBC will query DNS server to resolve NAPTR records over TCP connection.

In step 4, the SBC will try to send a NAPTR query and then immediately close TCP connection.

Expected result:

The NAPTR query should be successful in step.

The code is modified to close TCP connection (Sending FIN to close connection). Also, the connection information in application is freed.

Workaround: No workaround.

PortFix SBX-110746: The SIP ACK was missing in TRC, although the SBC sent one.

Impact: The 200 OK for the INVITE received on the egress call leg contains a Record-Route header with an FQDN. The SBC resolves the Record-Route FQDN through the DNS and routes the ACK to the resolved target. The SIP ACK PDU that the SBC sent to the egress call leg is missing in the TRC log.

Root Cause: The root cause is in SipsDnsLookupRspForTCB() when the DNS response comes back.
SIP_CALL_STR *pstCall is set as NULL when SipsTXNDownstreamMsgToFsm is called. Due to the NULL check of pstCall in later functions, the SipSgTraceDumpSipPdu() is not called to log ACK PDU in TRC log.

Steps to Replicate: 

1. Set up: SIP-SBX-SIP.
2. Create a dnsgroup and configure the external DNS IP Address. Attach dnsgroup with egress zone.
3. Create a global calltrace filter with "level1" and key "calling".
4. Start the global calltrace and roll trace log event.
5. Run SIP-SIP call. 200 OK for the INVITE received on the egress call leg contains Record-Route with an FQDN.
6. Verify that egress ACK PDU is logged in TRC log.

The code is modified so in the function SipsDnsLookupRspForTCB, SIP_CALL_STR* pstCall is fetched and use in SipsTXNDownstreamMsgToFsm.

Workaround: None.

PortFix SBX-110362: The SBC generates RTP inactivity alarms when the policer mode is set to bypass.

Impact: The SBC generates false RTP inactivity alarms when the policer mode is set to bypass.

Root Cause: A bug in the media policing logic of SWe_NP code skips updating the timestamp of last RTP packet of a media flow, resulting in raising RTP inactivity notifications for every 10 seconds.

Steps to Replicate: 

1. Configure a ISBC instance.
2. Set the policer mode to bypass (Unit tested by hacking the code).
3. Configure RTP inactivity.
4. Run the call for more than the rtp inactivity duration.
5. Observe that RTP inactivity alarms being raised, even if traffic flow is happening.

The code is modified to update the timestamp after every arrival of RTP packet for the media flow.

Workaround: Ensure the policer mode is set values to other than 'bypass".

Portfix SBX-109424: The SBC sends a 420 Bad Extension response when an INVITE with both supported and require 100rel is sent.

Impact: The SBC sends 420 Bad extension when Require:100Rel is received in initial Invite and e2e Prack flag is enabled

Root Cause: Incorrect code was added to reject an INVITE with Require:100Rel when rel100Support flag is disabled and e2e prack flag is enabled.

Steps to Replicate: 

1. The rel100Support must be disabled.
2. The e2e Prack is enabled.
3. An INVITE is received with a Require:100Rel Header.

The code is modified for rejection in the require:100rel scenarios.

Workaround: Use SMM to remove Require:100Rel Header.

PortFix SBX-90156: Observed the Major logs "MAJOR .CHM: *send_notification: unknown variable name sonusAlarmNodeID".

Impact: Major logs "MAJOR .CHM: *send_notification: unknown variable name" are seen on the sbxrestart.

Root Cause: These type of logs are seen when the queued traps are flushed out but the corresponding MIBS are not loaded yet, as a result unknown variable names/faulty varbind logs are seen.

Steps to Replicate: 

1. Spawn a OAM-MSBC setup as seen in the reported case and configure an SNMP trap target.
2. Perform a sbxrestart on the MSBC standby node.
3. Verify that the reported logs are not seen.
4. Verify the timeline for enabling of northbound interface from .SYS log and the timeline for flushing of queued traps from the level log and ensure the difference between them to be >=15 seconds.

The code is modified from ConfdSnmpStartupDelay from 5 seconds to 15 seconds to introduce a buffer for the loading of the MIBS, after the Northbound interface is enabled.

Workaround: Not applicable.

Portfix SBX-110201: A late offer call resulting in the SBC not sending DTMF for AMR-WB in initial offer towards ingress.

Impact: The SBC is not sending 16k 2833 Payload type in the initial offer towards ingress during a Late media "convert" call.

Root Cause: Answer received from egress contained both 8k and 16k 2833 Payload type and that resulted in the SBC incorrectly assigning the 8k PT value to 16k as well while generating offer towards ingress. As a result, the 16k PT get dropped by SIP stack.

Steps to Replicate: Configuration:

Transcode conditional
rel100Support enabled
honorSdpClockRate enabled
DLRBT enabled
Configure multiple codecs on both Routes

To re-create the issue:

1. The UE initiates Late media convert call.
2. The SBC sends out 180 answer with the below SDP:
   
   m=audio 1024 RTP/AVP 96 8 97 0 18 9 101
   a=rtpmap:96 AMR-WB/16000
   a=fmtp:96 mode-set=0,1,2; mode-change-capability=2; max-red=0
   a=rtpmap:8 PCMA/8000
   a=rtpmap:97 AMR-WB/16000
   a=fmtp:97 mode-change-capability=2; max-red=0
   a=rtpmap:0 PCMU/8000
   a=rtpmap:18 G729/8000
   a=fmtp:18 annexb=no
   a=rtpmap:9 G722/8000
   a=rtpmap:101 telephone-event/8000
   a=fmtp:101 0-15

Test Result without a fix:

The SBC drops the telephone-event/16000 payload type when generating offer towards ingress in 180.

The code is modified to prevent the same PT value getting assigned to both 8k and 16k DTMF in this call flow.

Workaround: None.

PortFix SBX-108112: MS TEAMS - The Media Stats are not correct when the DLRBT profile is removed from TGs and ICE is enabled.

Impact: On an SBC was configured for MS Teams LMO centralized mode with ICE, if DLRBT is not correctly configured, this can lead to media not flowing correctly for a call even after ice learning gets completed.

Root Cause: The early ICE learning logic for non DLRBT mode in the SBC was not fully deactivating ICE media resources on receipt of 200 OK . As a result, the resources were unable to be re-activated for end to end media flow.

Steps to Replicate: On an SBC configured as MS Teams LMO centralized mode with ICE on MS Teams TG:

1. Disable DLRBT on PSTN and MS Teams TG's.
2. Establish a PSTN endpoint to MS Teams call that uses primary (Internal) LIF address towards Teams.
3. Once call has established and ice learning has completed, send media (voice) in both directions between PSTN and Teams endpoints.
4. Media should flow as expected in both directions and call Media status should correctly show the number of media packets sent and received for ingress and egress.

The code is modified to correctly deactivate early ice learning media resources on receiving an SDP from MS Teams.

Workaround: The DLRBT should be enabled.

PortFix SBX-107999: The LeakSanitizer: CpxDnsCreate leaks observed during SBX-85432 E2E C-SBC automation run.

Impact: There is a small memory leak in the Cpx process when DNS elements are added or deleted.

Root Cause: As part of adding/deleting DNS entries the Cpx process reads in the interface group name from CDB and stores it in a local buffer but does not free it when finished processing.

Steps to Replicate: Add DNS server entries.

The code is modified to correctly free up the local buffer at the end of the configuration logic.

Workaround: None.

Portfix SBX-106759: In a N:1 mode, the SBC performs a switchover for a different node than the switchover request is honored for. 

Impact: In N:1 during few scenarios, SBC switchovers to node that was not request honored. When the issue occurred, a switchover will be processed to other node instead of processing to request honored node.

Root Cause: During switchover process SBC was not checking the node which was requested honored. It was switching over to a node that comes first/is available while processing in the SBC.

Steps to Replicate: 

1. Bring up a 2:1 SBC.
2. Try to reboot the both active server.
3. While switchover process delay the up of request honored node and allow the another node to come up first.

The code is modified to check the service id of the request honored node before a switchover.

Workaround: None.

PortFix SBX-105804: The CDR field is not populated even though the SBC writes the value to CDR field for '200 Ok of BYE' received/sent.

Impact: After SMM manipulation, the SBC writes the value to CDR field as seen in the DBG but the CDR field ‘STOP’ record is empty.

Root Cause: The logic to write the ACT file for the incoming 200 OK of Bye was absent.

Steps to Replicate: 

1. Attach the SMM profile to modify the 200OK of BYE on Egress TG as input Adaptor profile and output Adaptor Profile.
2. Enable endToEndBye flag under an ipSignalingProfile.
3. Run a basic A-B call.

The code is modified for updating the CDR field in the STOP record during the processing of response of BYE method.

Workaround: Attach the SMM to BYE instead of the 200OK of BYE.

PortFix SBX-111609: The GCID value of '100 Trying" sent is logged with value '0xffffffff' in POST-SMM SMM L4 Trace.

Impact: The POST-SMM Level 4 trace log for 100 Trying has GCID incorrectly set to 0xffffffff.

Root Cause: Level 4 call trace is active with a configuration to match 100 Trying and a SMM rule is applied that modifies 100 Trying.

Steps to Replicate: 

1. Configure a Level 4 call trace to match all messages.
2. Configure a SMM rule to modify the 100 Trying sent by the SBC.
3. Make a SIP-SIP call.

The code is modified so that the correct GCID value is printed in both PRE and POST-SMM traces.

Workaround: None.

PortFix SBX-106613: The SBC adds duplicate header on new INVITE upon 422 response, when a transparency profile is attached to egress TG.

Impact: The SBC adds a Duplicate Header on a new INVITE after the 422 Response is received.

Root Cause: The SIP Stack adds a semi-known twice (Header is unknown to SIP Parser but configured in transparency Profile) in this 422 scenario.

Steps to Replicate: 

1. Header has to be unknown to the parser.
2. The same header name needs to configured in the Transparency profile.

The code is modified to Skip formatting the second instance of same header based on header type.

Workaround: SMM can be added to remove the duplicate header.

Portfix SBX-112082: There was an runtime error: member access within null pointer of type 'struct CC_SG_ALERTING_UIND_MSG_STR in CcSgAlertHndl.

Impact: Run a basic G711U pass-thru call. After the call got completed, ASAN runtime error is observed in the system logs indicating that the code is taking the address of a field within a null pointer. This could potentially lead to coredumps if using the SIP recording capabilities other than SIPREC.

Root Cause: When a call Progress/Alerting message is received from the network, the code was taking the address of a field within the pointer.

Steps to Replicate: Run a basic call.

The code is modified to validate that the pointer is not null taking the address of a field within the pointer.

Workaround: None.

PortFix SBX-108434: SBC: A Standby OAM fails to come up after the restart of active and standby.

Impact: After restarting the active and standby or after a fault that causes the SBCs to go down, the standby OAM waits for active to come up first and never recovers if active is down.

Root Cause: On the Standby OAM startup sequence, there is a makeSureActiveIsUp() loop that exits only after active is up. This results in standby to be in a hung state forever if active is down.

Steps to Replicate: 

1. Start active OAM SBC.
2. Start the standby OAM SBC after 1 minute.
3. Stop the active OAM SBC before it becomes active.
4. The Standby OAM SBC should reboot and recover after 10 minutes.

The code is modified to reboot the instance that is starting as standby, if peer does not become active in 10 minutes.

Workaround: Reboot the standby OAM SBC to fix the issue.

PortFix SBX-110984: The EVS CMR not honored when in dtx period.

Impact: If the SBC receives a CMR request when it is operating in DTX mode, the CMR was not being processed.

Root Cause: The rate or bandwidth change received through a CMR was not put into effect after coming out of the no transmission period.

Steps to Replicate: Run an EVS<=>EVS call with dtx enabled. On the Ingress leg, send a valid CMR while there is no media being sent on the Egress Leg. Send a CMR on the ingress leg when it is in the "no transmission period"

Once the Ingress Leg comes out of the no tx period, the bitrate or the bandwidth as requested by the CMR should be put into effect.

The code is modified to address the issue.

Workaround: None.

PortFix SBX-109304: The 13.2 wb cmr is not accepted when the SBC is operating in channel aware mode.

Impact: The channel aware mode once enabled will always be enabled when the EVS encoder operates at 13.2Kbps and bandwidth WB. The only way to disable channel aware mode is to use a bitrate other than 13.2Kbps.

Root Cause: The code to disable the channel aware mode if enabled on receiving on 13.2 WB CMR was absent.

Steps to Replicate: Run a EVS to G711 call with br=13.2, ch-aw-recv-5; bw=wb.
Stream a pcap having CMR for 13.2 WB from the peer.

The call starts with 13.2 Kbps with Channel Aware mode being enabled. On receiving the CMR, the channel aware mode will be disabled while the bitrate is still 13.2.

The code is modified to address the issue.

Workaround: None.

PortFix SBX-109300: The SBC should not accept cmr byte from discarded packets.

Impact: The SBC was accepting the CMR from discarded packets.

Root Cause: The CMR was being processed before Packet Validation.

Steps to Replicate: Run and EVS<=>EVS call. Let the peer send 32Kbps packets having CMR for 24.4Kbps.

In this case, the 32Kbps packets are discarded as we do not support transcoding beyond 24.4Kbps for EVS. Also since the packets are discarded CMR for 24,4Kbps is also not honored.

The code is modified to validate the packet first followed by the CMR processing.

Workaround: None.

PortFix SBX-110956: Introduced the Upsampler for EVS in a EVS(wb)<=> NB codec scenario.

Impact: The WB will not be used for EVS encoding and as a result, the channel aware mode cannot be enabled in an IDP NB scenario though negotiated through the SDP.

Root Cause: There is an absence of an Upsampler in the Upstream path for EVS when WB is negotiated in an IDP NB scenario.

Steps to Replicate: Run a EVS<=>G711 call with bw=wb; ch-aw-recv-5;br=5.9-13.2.

In this case, the EVS encoder produces WB packets with channel aware mode enabled.

The code is modified for the EVS when the WB is negotiated in an IDP NB scenario.

Workaround: None

PortFix SBX-111721: The EVS encoder picks up the initialCodecMode as the bitrate after switch to Compact Format.

Impact: The EVS encoder picks up the initialCodecMode as the current bitrate if a change in packet format from Header Full to Compact is triggered.

Root Cause: The root cause is the re-initialization of EVS encoder with bitrate as the initialCodecMode on change in packet format.

Steps to Replicate: 

1. Run a EVS<=>AMRWB call.
   br=5.9-24.4; bw=nb-wb
2. Stream a WB pcap with 13.2Kbps packets with CMR byte for 13.2 CA mode for the first 10s. After, the pcap has 24.4 Kbps packets without the CMR.

Results:
The mode of operation changes from 24.4Kbps to 13.2 Kbps with CA enabled on receiving the CMR. Once the Endpoint starts sending 24.4 Kbps packets without CMR, the SBC switches to Compact Mode while maintaining its bitrate as 13.2 Kbps with CA enabled.

Prior to the fix, when switching to the Compact mode, the SBC would use 24.4Kbps (initialCodecMode) as the bitrate.

The code is modified to the re-initialize the EVS encoder with bitrate as the localCodecMode rather than the initialCodecMode.

Workaround: None.

PortFix SBX-106520: Among the exported setting values, there were setting values that were not set in other environments.

Impact: The "comment" parameter is missing from some configuration related to AAA rules/cmdRules.

Root Cause: The "comment" parameter is not being restored during a LSWU.

Steps to Replicate: 

1. Spawned a SBC, exported the configuration and observed comment parameters are present.
2. Upgraded the SBC software. exported the configuration and observed comment parameters are not present.
   
   Revised the steps above with fix, and the issue was not observed.

The code is modified to restore "comment" parameter during a LSWU.

Workaround: None. The "comment" parameter is used only for description. It does not have functional impact.

Portfix SBX-109209: SBC: Observed MAJOR logs for SIPFE "/localstore/ws/jenkinsbuild/sbxMain/marlin/SIPFE/sipFeSigPortCsv.c, SipFeGetSipSigPortStatisticsGetNextReqMsg, 1111] Another Query in process" on T140 load.

Impact: Observed MAJOR logs for SIPFE "/localstore/ws/jenkinsbuild/sbxMain/marlin/SIPFE/sipFeSigPortCsv.c, SipFeGetSipSigPortStatisticsGetNextReqMsg, 1111] Another Query in process" on T140 load.

Root Cause: The addressContext zone sipSigPortStatistics table is not supported on the SBC, so the code used to return the information was sometimes not returning, causing the error messages above.

Steps to Replicate: 

1. Run the following command: 
   snmpwalk -c admin -v2c <mgmt address of sbx> 1.3.6.1.4.1.2879.2.10.2.121
   Repeat this command 4 times.
2. Verify the following error:
   "/localstore/ws/jenkinsbuild/sbxMain/marlin/SIPFE/sipFeSigPortCsv.c, SipFeGetSipSigPortStatisticsGetNextReqMsg, 1111] Another Query in process" on T140 load
   Does not occur in the DBG log.

The code is modified so SipFeGetSipSigPortStatistics routines return immediately.

Workaround: Do not query that table.

PortFix SBX-108620: A runtime error: load of value 24752, which is not a valid value for type 'SIP_MSG_TYPE'.

Impact: While processing an in dialog NOTIFY message, the check for message type was not done using proper data type.

Root Cause: Message type check for NOTIFY message was not correct.

Steps to Replicate: 

1. Make a call on an SBC installed with ASAN build.
2. Send an in dialog notify.
3. The error above should not be displayed in ASAN logs.

The code is modified to address the issue.

Workaround: None.

Portfix SBX-108232: SBC: The AddressSanitizer: detected a heap-use-after-free on address 0x61800000a5a8 at pc 0x559e8989c4ac bp 0x7f4411fde290 sp 0x7f4411fde288 READ of size 8 at 0x61800000a5a8 thread T9.

Impact: While the SBC node is shutting down it can access memory after its been freed, this could result in unexpected behaviour and in the worst case a coredump. But would have limited impact as it only happens when shutting down.

Root Cause: During a sbxstop/sbxrestart or switchover because of race-condition, when the SBC is in deactivation the oamNodeRegisterRetry can access already deallocated resource leading to a core dump.

Steps to Replicate: 

1. Setup build with HA SBC using OAM.
2. Perform a sbxrestart/switchover of active instance.

The code is modified to handle this race condition.

Workaround: None.

PortFix SBX-108411: [ASAN]: sanitizer.CE_2N_Comp_ScmProcess_2.30828:==CE_2N_Comp_ScmProcess_2==30828==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x61b00008dc88 at pc 0x5621c53c6946 bp 0x7f9854100bc0 sp 0x7f9854100bb8.

Impact: While running a INVITE CANCEL Proxy Call, the UAS observed a Address Sanitizer leak in SCM process and the SBC services stopped.

Root Cause: Issue in parsing string when string is blank and very large that lead to a heap buffer overflow.
Example: SIP/2.0/UDP 10.xx.xx.xx:7009;sigcomp-id=" LWS "

When the token length becomes large 100 say (which is the LWS length) and this token length point outside the PDU, we will see this issue.

Steps to Replicate: Send an INVITE with a VIA header as the last header.

The code is modified to use temptokLen instead of the tokLen. As a result, the tokLen does not reach outside the PDU.

Workaround: The VIA header should not be last header, and the empty token string should be small.

PortFix SBX-111270: The CDR record for DSP insertion/rejection reason field in 9.2.2R000 test execution has marked the call as “Transcoding” instead of “Transcoding due to DTMF”.

Impact: The CDR DSP insertion reason is showing as trancoded instead of the DTMF for transcoded call with difference in DTMF parameters on the ingress and egress leg.

Root Cause: The wrong code is present and overwrites the DSP insertion reason as transcoded.

Steps to Replicate: Make a successful transcoded call with AMR codec on both sides and the
DTMF parameter must be different in both ingress and egress.

The code is modified to set the DSP insertion reason based on the answered call leg.

Workaround: None.

PortFix SBX-110439: The DSP rejects CMR requesting channel-aware if localCodecMode is set to a value other than 13.2.

Impact: The SBC was rejecting a Channel Aware Mode CMR when the current mode of operation of EVS encoder was not 13.2Kbps WB.

Root Cause: A conditional check of the current Bandwidth and current Bitrate to honor a Channel Aware Mode CMR.

Steps to Replicate: Run a EVS to AMRWB call with the following SDP:

br=13.2-24.4; bw= nb-wb; ch-aw-recv=0

End point sends a CMR for Channel Aware Mode.

In this case, the call starts with EVS encoder encoding packets as 24.4Kbps and on receiving the CMR, the rate changes to 13.2Kbps with Channel Aware mode enabled.

The code is modified to check if 13.2Kbps is a part of the activeCodecSet and whether WB is present in the bandwidth range negotiated to honor a Channel Aware Mode CMR.

Workaround: None.

PortFix SBX-109221: The dpf_core.c "runtime error: shift exponent 432 is too large for 64-bit type 'long long unsigned int'" in np.log.

Impact: In the ASAN build, a runtime warning is observed in SWe_NP while running AMR-EVS audio call and insert T140 to both the streams through a RE-INVITE.

Root Cause: Incorrect use of bitmask operation to extract dscp field from packet.

Steps to Replicate: In an ASAN build, make a AMR-EVS audio call and insert T140 to both the streams through a RE-INVITE. AMR+T140-> EVS+T140. Observe ASAN warning in the np.log.

The code is modified to address the issue.

Workaround: None.

PortFix SBX-110299: The SBC does not active EVS partial redundancy mode during the call when the remote offers "ch-aw-recv=0"

Impact: When the ch-aw-recv=0 is negotiated through the SDP, a CMR request for the Channel Aware mode with offset 2 and the priority HIGH was not being processed.

Root Cause: The root cause of this issue was a wrong initialization of the channel aware mode offset and priority in the code.

Steps to Replicate: 

1. Run a EVS to G711 call with br=13.2; bw=wb;ch-aw-recv=0
2. Stream a pcap with CA CMR for priority HI and offset 2.

Prior to the fix the CMR is not processed. With the fix, the CMR is processed and also honored.

The code is modified to address the issue.

Workaround: None.

PortFix SBX-107396: Error "SYS ERR - Error 0x3b Line 666" was being printed frequently.

Impact: SYS ERR repeatedly logged in SYS logs.

Root Cause: The SYS ERR seems to only occur for audio encoding type 0x3b, which is SPEEX_8. But we do not have enough information to determine the call flow that triggered it.

Steps to Replicate: The steps cannot be reproduced. 

The code is modified to replace the SYS ERR with a major level debug message with the GCID and audio encoding type to help future debugging.

Workaround: None.

The IPSEC Tunnel down after running an SBC Upgrade to 8.2.2R1.

Impact: After the upgrade is completed, during IPSec SA rekey, the IPSec SA was not getting established.

Root Cause: After an upgrade, when IPSec SA gets rekeyed the newly created SA is not getting configured into Kernel and hence IPSec SA creation timeout happens.

Steps to Replicate: 

1. Establish IPSec SA, send traffic over tunnel.
2. Perform an upgrade.
3. Wait for IPSec SA expiry and rekey.
4. Once rekey is complete, check that new SA got created after rekey and traffic is sent over tunnel.

The code is modified so that the new IPSec SA created during rekey after upgrade is configured into Kernel and the IPSec SA is established without any issues.

Workaround: Perform an SBC switchover to help clear the problem.

The SBC uses port number RTP+1 for RTCP instead of the learned RTCP port number if the RTCP NAT learning completes before RTP NAT learning.

Impact: The SBC sends the RTCP packets to a destination port number RTP+1 for the RTCP instead of the learned RTCP port number if the RTCP NAT learning completes before RTP NAT learning.

Root Cause: The SBC overwrites the learned RTCP port number with the RTP+1 port number if the RTCP is learned before RTP.

Steps to Replicate: Send RTCP packets first then RTP packets to the SBC. After call is connected, verify the RTCP port number, if RTCP is learned before RTP.

The code is modified to use correct RTCP learned port when RTCP learning occurs before RTP, instead of comparing the RTP and RTCP addresses from callLeg structure.

Workaround: No workaround.

PortFix SBX-109103: There was an incorrect UDP port used in the Record-Route and Contact on the core side.

Impact: The SBC populates an invalid port in the Contact/Record-Route header towards the IMS Core side in call progress (180/200) responses.

Root Cause: The signaling engine in the SBC, during a call, progress command modifies the ports without validating the direction of message flow and causes the SBC to populate an invalid port in the SIP responses.

Steps to Replicate: 

1. Send an IMS-AKA registration request from a UE device.
2. Ensure the IMS-AKA registration is successful.
3. Let the registered user receive call from IMS network.
4. When UE accepts the call, we can observe in the call progress response sent from the SBC towards the IMS Core will have the invalid port in contact/Record-Route headers.
   
   Flow: IMS Core --(SIP UDP)-> SBC ---(SIP IPSec TCP)--> Peer (UE)

The code is modified to identify the direction before changing ports, change only when the flow is towards the UE.

Workaround: Use the SMM to change the ports in Contact and Record-Route headers.

The values in callsEstimate.txt differ between the Active and Standby SWe.

Impact: The session capacity estimate values differ in active and standby VMs upon an LSWU upgrade of 1:1 the SWe HA pair.

Root Cause: The following sequence of events resulted in the issue:

1. The processor indices are re-calculated upon LSWU upgrade, which differs from the processor indices calculated by the SBC VM in the earlier version.
2. The session estimates are calculated based on these new processor indices.
3. Post upgrade, once the application comes up, the processor indices that are stored in DB(from the older version VM), are restored back to the /opt/sonus/conf/swe/capacityEstimates/.index.txt file.

As a result, the Active and Standby VMs obtained different session estimates upon upgrade.

Steps to Replicate: After subjecting the 1:1 SWe HA to LSWU upgrade, content of the following file in Active and Standby VMs would differ:
/opt/sonus/conf/swe/capacityEstimates/.callsEstimate.txt

The code is modified to retain the processor indices during the LSWU procedure.

Workaround: After completing the LSWU upgrade, once the active (VM-A) and standby(VM-B) VMs are in sync, following a set of operations would make sure that session capacity estimates are identical in Active (VM-A) and Standby(VM-B) VMs:

1. Reboot the Standby (VM-B) VM.
2. The standby (VM-B) VM comes up with correct session estimates. Wait for the completion of application sync.
3. Now, reboot the Active (VM-A) VM. This results in application failover. Application running on the Standby(VM-B) VM takes over the active role.
4. Active (VM-A) VM comes up with the correct session estimates and comes in sync with the application running on the Standby (VM-B) VM.

If the VM name in the upper right corner of the "Configuration Manager" EMA/EMS SBC Manager is clicked, the browser freezes.

Impact: If the VM name in the upper right corner of the "Configuration Manager" EMA/EMS SBC Manager is clicked, the browser freezes.

Root Cause: The issue occurred because the peer setup was unreachable and to get the peer system info, the curl command that is executed was taking default timeout to get a connection timeout.

Steps to Replicate: 

1. Log into EMS.
2. Launch the SBC Node.
3. Click on Monitoring -> System and Software info tab.

The code is modified to address the issue.

Workaround: Refresh the UI page.

The 822R0 SCM Process core dumped.

Impact: The SCMProcess core dumped on a SWe.

Root Cause: Active SCM sent a Redundancy message to Standby SCM with a Registration index that was outside of its MAX range. The error handling code caught this and attempted to clean up the Registration Control Block but had a problem because the Control Block had not yet been initialized.

Steps to Replicate: This problem is not reproducible.

It is most likely triggered by a race condition which results in the Active and Standby to have different numbers for max number of Registrations (on a SWe the max number of Registrations comes from the file callEstimates.txt and in this case, the files on the active and standby do not match).

This appears to be a SWe specific issue (the callEstimates.txt file is only used in SWE setups).

The code is modified to initialize the Registration Control Block immediately after allocating it, before any validation checks, so that the error handling code can clean up the RCB without causing a core.

Workaround: There is no known workaround.

The IMS preconditions are failing due to UPDATE message race condition.

Impact: An UPDATE from the calling party is queued by the SBC because the SBC is waiting for 200 OK for the previous update sent to called party, and when the 200 OK is received, the SBC did not forward the queued Update.

Root Cause: Precondition parameters received as part of UPDATE from ingress endpoint is not updated in the egress CCB.

Steps to Replicate: 

1. Configure preconditions to transparent on both legs.
2. Configure transmitpreconditions to supported on both legs.
3. Run the script so that there is difference in precondition parameters in an INVITE and UPDATE messages from ingress.
   
   invite: a=des:qos mandatory local sendrecv
   a=curr:qos local none
   a=des:qos optional remote sendrecv
   a=curr:qos remote none
   
   (AS script)183 in Progress: a=curr:qos local sendrecv
   a=curr:qos remote none
   a=des:qos optional local sendrecv
   a=des:qos mandatory remote sendrecv
   a=conf:qos local sendrecv
   
   (IAD script)update : a=curr:qos local sendrecv
   a=curr:qos remote sendrecv
   a=des:qos mandatory local sendrecv
   a=des:qos optional remote sendrecv
4. Send the update from the Ingress when the SBC is waiting for 200 of the previous update it sent to egress.

Verify that the SBC sends the queued update, once it receives the 200 ok for the 1st update.

The code is modified so the precondition level of support was set to transparent on both legs in configuration.

When the SBC receives precondition parameters as part of an UPDATE and preconditions are transparent, then the SBC saves the precondition parameters into the egress CCB. When the queued update message is being processed there is check for preconditions flag and comparison of precondition variables is done to send the update to called party.

Workaround: Enable the "Disable media lockdown" flag in IPSP for egress leg so that the SBC does not send the first update.