SBC Core 09.02.02R002 Release Notes

PortFix SBX-111688: The Request-URI and TO fields in a SIP INVITE are incorrect and overwritten.

Impact: When the Diversion header presented in an Ingress INVITE and had no RN in Request-URI, the egress RURI would use the RN in username field, after a PSX LNP dip.

Root Cause: The fix for a previous issue broke the previous RURI username setting.

Steps to Replicate: 

1. Set up the SBC with an external PSX to make LNP calls.
2. Pay attention to the disableRn flag in egress IPSP in PSX and flag UseGAPwhenRnisDisabled in the SBC's egress TG. Test different combination of < disableRn, UseGAPwhenRnisDisabled >
3. Ingress INVITE needs to contain Diversion header and no RN in the Request-URI.

Sample message could be found in the reproduced DBG logs and finalFix DBG logs.

The code is modified to ensure the RURI always contains the correct username.

Note: When the egress IPSP has "SIP TO Header Mapping" set to "Called Number", the TO header's username is the ingress TO URI. The number will not be globalized. If the customer would like it to be globalized, the globalization profile of the egress TG could be modified to globalize TO URI. Another way is using SMM to modify it.

Workaround: Use the SMM and use the Warning-21-00029918

Portfix SBX-111223: Memory leak since upgrading to 8.2.5R0.

Impact: High memory utilization.

Root Cause: Two leaks of the same structure exist:

1. A memory leak is seen when a relayed SUSCRIBE message is cranked back and an Alternate Server cannot be found.
2. The code that handles relayed messages may cause a leak in certain error scenarios.

Warning-21-00029922 pertains to this issue.

Steps to Replicate: This is a memory leak that may be triggered if a relayed SUBSCRIBE is cranked back and then an alternate route is not found.

1. This memory leak that may get triggered if a relayed SUBSCRIBE is cranked back and then an Alternate Server is not found.
2. We were unable to reproduce this error scenario which causes a leak of the Relay Control Block.

1. The code is modified to take the correct path when an Alternate Server cannot be found while handling a relayed message that has been cranked back.
2. Code has been added to start a timer when we begin processing a relayed messsage. In error cases, the Relay Control Block will be free when the timer expires - preventing the structure from leaking.

Workaround: None.

Once the memory utilization reaches 91%, an automatic switchover occurs. To avoid an automatic switchover, Ribbon recommends performing a manual switchover during a low traffic period. 

An LSWU on SWe (VMware/KVM) through the Platform Manager fails with the additional reboot of the standby.

Impact: An LSWU from 09.02.02R001 to any higher release 1:1 HA SWe (KVM/VMware) through the Platform manager fails because the standby instance undergoes an additional reboot during the upgrade procedure.

Root Cause: The Index Marker file was missing on the standby instance prior to the LSWU procedure.

The Index Marker file was missing on the standby instance due to when the application comes up with the standby role, the Index Marker is created only when there is a mismatch in the calculated and DB populated indices.

Steps to Replicate: 

1. Create 1:1 HA SWe on KVM/VMware.
2. After the HA application comes in sync, run clearDB on the standby.
3. Perform an LSWU on the 1:1 HA SWe.

The code is modified so the Index Marker file is created on the standby instance irrespective of processor index mismatch in the calculated and DB populated indices.

Workaround: Before initiating the upgrade procedure (from PM or CLI), the user needs to run the following command as a root user from the linux shell of the active and standby instance.

touch /opt/sonus/conf/swe/capacityEstimates/.indexMarker

The term-ioi is not be set in the STOP record for a customer call flow.

Impact: When running a JJ90.30 to JJ90.30 call flow, the term-ioi value from the PCV header is not being stored in the ingress protocol variant string when the transitPCV control in the JJ90.30 interworking profile is enabled.

Root Cause: The original development work was only storing the term-ioi value in the egress protocol variant string.

Steps to Replicate: 

Make JJ90.30 to JJ90.30 call.
transitPcv is set to enabled in the customer's Interworking Profile.

Call 1:

INVITE --> (PCV with icid-value and orig-ioi)
<--- 180 (PCV with icid-value, orig-ioi, term-ioi)
<-- 200 OK (PCV with icid-value, orig-ioi, term-ioi)
--> BYE

Call 2:

INVITE --> (PCV with icid-value and orig-ioi)
<--- 180 (PCV with icid-value, orig-ioi, term-ioi)
<-- 200 OK (PCV with icid-value, orig-ioi, term-ioi)
<-- BYE

After a call, verify the CDR records. In both START and STOP records, the term-ioi is filled in ingress PVSD.

The code is modified to ensure the term-ioi value is stored in the ingress protocol variant string.

Workaround: None.

PortFix SBX-112381: The SBC had a Crash-Application in the rebooting loop.

Impact: A PRS Process core is occurring when the code is processing an ICE STUN packet and there are more than 20 Teams clients ringing. This could potentially happen in a simultaneous call group pickup scenario.

Root Cause: The root cause of the core is a bug in the code that handles the incoming STUN packet. This code is overwriting the end of array by writing too many entries into the array. This results in memory corruption and eventually a core.

Steps to Replicate: Run a call group pickup scenario with more than 20 users all having their Teams clients ringing at the same time.

The code is modified to prevent it from writing too many entries into the array.

Workaround: Disable media-bypass or reduce the number of users in the call group.

The SCM Process is coredumping.

Impact: The SBC is relaying multiple reason header instances in the amount of the quantity squared.

Root Cause: Logical error when multiple instances of a reason header on the same line. The SBC is relaying in the amount of quantity squared. 

Steps to Replicate: 

1. Configure the transparency Reason header.
2. Make a SIP-SIP call, Ingress sends BYE with multiples Reason header instances on the same line. The SBC sends instances in an amount of the quantity squared to the Reason header on Egress.

Note: If the ingress sends a large number of instances on the same line.

• For old (pre 8xx), the SBC may core.
• For 8xx and above, the SBC may be unable to send BYE on Egress.

The code is modified to address the issue.

Workaround: Disable the transparency of a reason header.

A restore on revision 1 failed in the standby OAM node.

Impact: A restore on revision 1 failed on the standby intermittently.

Root Cause: When the restore revision is requested, both OAM nodes restart. In a corner case, there is possibility of both starting as active and later one of the nodes will go down and not come up as standby.

Steps to Replicate: 

1. Create a scenario with 1:1 OAM for the SSBC/MSBC/MRFP.
2. In automation, keep the cleanStart as 1 so that the automation will trigger restore revision 1.

Both OAM should come up.

The code is modified to handle the corner case.

Workaround: Manually reboot the standby.

Uploaded onfig backup file is not available at gconfig.

Impact: The uploaded config backup file is not available on the gconfig.

Root Cause: The uploaded file was not moved in the config directory because the system name was not correct.

Steps to Replicate: 

1. Log in to the EMA.
2. Go to Administration -> System Administration -> File Upload.
3. Upload the file.

The code is modified to read the correct system name.

Workaround: None.

The SFTP transfer is not working to CDR server that only supports AES128_CTR.

Impact: One customer has a CDR server that offers AES128_CTR for the server-to-client encryption. When the SBC offers AES256_CTR as the preferred option, the SBC encodes the packets with AES256_CTR, and receives the packets with AES128_CTR encryption. This scenario causes a decoding issue on the SBC that results in the termination of the connection.

Root Cause: An invalid encrypted data length is received from the CDR server.

Steps to Replicate: 

1. Rollover the accounting file.
2. Verify that the CDR transfer is successful.

The code is modified to offer up the AES128_CTR as the preferred encryption option, so both sides are using AES128_CTR.

Workaround: Disable the AES256 on the CDR server.

PortFix SBX-112267: The correct 92x serialization results in test failures.

Impact: When the registration control block is made redundant and the active/standby instances are running different software releases, the redundancy logic might not work correctly if the registration control block contains P-Charging-Vector (PCV) and/or P-Charging-Function-Addresses (PCFA) information. The redundancy logic will work correctly post upgrade when the serialization of the redundancy data is not required.

Root Cause: The parameter lengths of the PCV and PCFA redundant parameters was not calculated correctly. This can lead to the redundancy code being unable to process all the redundancy information correctly.

Steps to Replicate: Run registration related tests with PCV and PCFA header parameters in an older release and then upgrade to 922R2 or later.

The code is modified to correctly set the length of these parameters to avoid problems with redundancy.

Workaround: None.

PortFix SBX-111659: The intercept is not working when P-Com.Session-Info is received in INVITE and tones are enabled.

Impact: When the lawful intercept target is specified in the P-com-session-info header of the INVITE message, the intercept does not occur if an announcement resource is applied to the call and then freed up. There is no issue if the P-com-session-info header is received in the backward direction.

Root Cause: This occurs when using P-early-media with tone profile and media monitoring profile applied to the call flow and the B-party sends P-early-media with a=inactive to start the media monitoring and does not provide RTP or does not send 200 OK prior to the monitoring timer expiring which results in the SBC playing RBT via an announcement resource. When the media path is later cut through and the announcement resource is released the intercept information was mistakenly freed up and the intercept did not occur.

Steps to Replicate: 

1. Run a lawful intercept call flow using P-com-session-info header in the INVITE to indicate the target intercept point. The call should be configured with P-Early-Media, tone profile and monitoring profile to check for RTP packets received.
2. In response to the egress INVITE, send back an P-early-media header with a=inactive and no RTP packets prior to the monitoring timer expiring so the SBC plays RBT.
3. Answer the call and check that media is being sent to the intercept server.
   

The code is modified to ensure the intercept information is not freed up when freeing the announcement resource.

Workaround: None.

The LI connection to media server was not connected on a switchover.

Impact: When using the PCSILI (P-com-session-info flavour of LI) in an N:1 M-SBC setup and running the SBC release 8.2 or later if there is a switchover, the connection to the LI server might not be re-established.

Root Cause: The standby instance was trying to maintain information about the operational status of the interface used for LI processing. But this was only happening correctly for the first instance. The status of the other instances was being misinterpreted and lost. So during the transition to active, the LI code thought the interface was not available and did not try to establish the connection to the LI server.

Steps to Replicate: In an N:1 setup perform switchovers from each active instance to the standby and check that the connection to the LI server is restored.

The code is modified to collect the status of the interface after the instance transitions from standby to active so that the accurate interface information is available for LI processing.

Workaround: Bounce the interface e.g. ifup/ifdown that is used for LI connection would help to recover the connection.

PortFix SBX-108317: The SBC switchover and isolation of node/SVWSBCD.

Impact: The PRS core was truncated because a second ABRT was sent to the process.

Root Cause: The PRS core was truncated because a second ABRT was sent to the process while it was in the process of writing the core.

Steps to Replicate: This issue is not reproducible.

The code is modified to prevent two ABRT signals from being sent to the same process.

Workaround: There is no workaround.

PortFix SBX-111050: There was a coredump during a reboot on a customer SBC. 

Impact: A misconfiguration of GW Signaling Ports, in which there is a Secondary GW Signaling Port configured but no Primary GW Sig Port configured, and can lead to a core and switchover.

Root Cause: When there is no Primary GW Signaling Port configured but there is a Secondary Signaling Port configured, we enter a code path that attempts to dereference a NULL pointer (pointer to Primary Sig Port) when it is attempting to get the dscp value to use for the socket.

Steps to Replicate: An SVT engineer should:

1. Configure an SBC with a Secondary GW Signaling Port without configuring a Primary GW Signaling Port.
   i.e. gwSigPort of SBC1 should be in primary mode and SBC2 should be in secondary mode

2. Send a GW-GW call from a remote GW through this SBC using the address of the Secondary GW Signaling Port on an SBC.

   Without the fix, this should cause a core.
   With the fix, there should be no issue.

The code is modified to prevent it from attempting to dereference a NULL pointer (pointer to Primary Sig Port) when it is looking up the dscp value to use for the socket.

Workaround: The workaround is to avoid configuring a Secondary GW Signaling Port without configuring a Primary GW Signaling Port.

PortFix SBX-110199: There was dbg log flooding with MAJOR .VNFR: *(VAgent) message

Impact: Event Logs are flooded with error message, when one of the OAM node or Managed VM goes down.

Log Message:.MAJOR .VNFR: *(VAgent): send error. rc=-1, error=Resource temporarily unavailable

Root Cause: The health check message send fails, when an instance is down.

Steps to Replicate: Run an LSWU from VNFM on an ISBC SWE instance in 1 to 1 HA mode.

The code is modified so the messages are only logged on the first error in health check. It is reset, when a message send succeeds.

Workaround: The error message stops automatically once the node is recovered and starts to respond to the health check messages.

PortFix SBX-110884: The logs are getting printed in openclovis.

Impact: If condition present, SCM is filling openclovis logs with this error message:
nprintf buffer too small. Need 306 Have 300 File: /sonus/p4/ws/release/sbx5000_V08.02.02R001/marlin/SIPSG/sipsgRegAgent.c Line: 2527

Root Cause: The error message is being logged because the code is attempting to write a debug log into a local buffer that is not big enough.

Steps to Replicate: No testing is required. This is no risk with/without fix.

The code is modified to increase the size of the local buffer.

Workaround: There is no workaround.

PortFix SBX-109243: The SBC sending 481 for CANCEL in dialog transparency.

Impact: When the dialog transparency is enabled and call loops back to the SBC, the SBC does not handle a CANCEL sent from ingress endpoint.

Root Cause: The SBC expects a CANCEL to have callinfo header.

Steps to Replicate: 

1. Reproduce the issue.
   Dialog Transparency is enabled and the Call Loops back In.
   After adding PRACK, Ingress sends CANCEL without callinfo header.
   The SBC sends a 481.
2. With a fix, the SBC sends 200OK for CANCEL and relays to egress leg.

The code is modified to ensure the SBC finds the correct SG to handle the CANCEL even when the CANCEL does not have callinfo

Workaround: None.

Host check validation failing - Required GB RAM 6 but found 0.03125.

Impact: The few host machines in the AWS data center use different units for RAM, HostCheck script assumes that available memory is given in MB. The HostCheck script is fixed to calculate available memory based on unit of memory.

Root Cause: The HostCheck script does not have code to consider memory unit.

Steps to Replicate: Create VM with >=32 GB RAM, application should come up without complaining about memory in VM.

The code is modified to calculate available resources based on units.

Workaround: Create instance/VM that has < 32 GB RAM.

Portfix SBX-107133: Max FD limit is reached in SLB beyond 7,04,000 TLS connections (Access Registrations) tried.

Impact: Max FD limit is reached in the SLB beyond 7,04,000 TLS connections (Access Registrations) tried.

Root Cause: Observing the FD congestion at high load due to FD limits reached.

Steps to Replicate: Check the updated FD limits in /etc/security/limits.conf.

The code is modified for the max FD for the KVM and SLB.

Workaround: None.

The SBC is forming invalid packet where it is adding 00 in around all headers and parameters.

Impact: The SBC puts a NULL termination in every parameter and header of the message. In this case, there was a parser error on a parameter when the DNS translation was required.

Root Cause: During the parsing of the message after a DNS query, if a parser error occurs, then incorrect termination is put/left in the message when sent on the wire.

Steps to Replicate: 

1. An incoming INVITE had a known syntax error related to alert_info, configuration was to parse through the filterProfile and not transparently forward alert_info. This resulted in parse error and bad formatting when DNS was executed.
2. Configure the alertInfo to transparently pass on egress, and DNS function doing fqd/IP swap executes correctly and egress message is sent with good formatting.

The code is modified to:

1. Log parse an error line number and pdu message.
2. Apply the filterProfile when parsing the message. The customer needed to add filterProfile to transparently forward the parameter failing parsing on egress to avoid parse error and avoid incorrect parameter termination.

Workaround: On the Ingress, apply an SMM to rename alert_info to an unknown (x-alert_info), and rename back on the egress.

The file upload is not working in the Platform Manager.

Impact: The file upload is not working in the PM.

Root Cause: The request URL length was too high as per-server configuration because that requested call was failed.

Steps to Replicate: 

1. Log in to the Platform Manager.
2. Go to Administration -> System Administration -> File Upload.
3. Upload a file and save it.

The code is modified to make a request call successful.

Workaround: We can apply the same changes in a customer setup also and restart the apache server.

Getting badRidCb errors during LSWU from 7.2.4R0 to 9.2.2A003

Impact: An upgrade from an older version to 9.2.0R0 and newer, the NP reported badRidCb error continuously: MAJOR .IPM: *NP 0 error counter badRidCb incremented: cnt 2310 last error 0x10002.

Root Cause: In version 9.2.0R0, Ribbon Protect streaming support was added. During an LSWU, new fields related to Ribbon Protect streaming were all initialized to zero in internal data structures, including the rbbnType. But in NP, rbbnType = 0 triggered NP to send RTCP_APP related statistics to Protect server, which caused the errors.

Steps to Replicate: Run an LSWU from any older version to 9.2.2 with call loads.

The code is modified to initialize the rbbnType to 'BRM_RBBN_PROTECT_STREAM_DISABLE' so that NP does not generate RTCP_APP to Protect server.

Workaround: No workaround.

PortFix SBX-111211 to 9.2.x - Get "violates foreign key constraint" error when assigning timeRangeProfile to a Route in EMA

Impact: Route creation fails when the name of time range profile contains numerals.

Root Cause: During the route creation, validation of time range profile was failing as it was containing numerals.

Steps to Replicate: 

1. Create a time range profile with a name containing number ex test_1
2. Create a route with test_1 as time range profile.
3. Route creation should be successful.

The code is modified to consider numerals as valid a character.

Workaround: Create a Route from the CLI.

PortFix SBX-108616: Fora  Late Media Call, the SBC is not sending a second UPDATE towards ingress when the DLRBT is enabled.

Impact: In a late media convert call scenario, when the DLRBT is enabled, the SBC is not sending UPDATE towards the ingress with the list of codecs received from UAS.

Root Cause: The minimizing of media changes from other call leg functionality is suppressing the triggering of the UPDATE towards UAC.

Steps to Replicate: 

1. The UAC sends INV without SDP to the SBC.
2. The SBC sends INV with PCMA PCMU G729 101.
3. The UAS sends 180 with PCMA G729.
4. The SBC sends 180 with PCMA G729.
5. The UAC sends PRACK with G729.
6. The SBC starts playing tone with G729 towards ingress EP.
7. The UAS sends 200 OK (for INV) without SDP.

The code is modified to identify this particular call scenario, such that, the SBC sends an UPDATE towards the UAC if the tone codec is different from the end to end negotiated codec.

Workaround: None.

Portfix SBX-111502: Metavar exchange continues even after the allocation is failed in the SBC.

Impact: The CHM processes the metavar request/response messages even when the application is going down, which results in failure messages.

Root Cause: The CHM does not check if the application is going down while processing the messages from the other nodes.

Steps to Replicate: Bring up the SBC in N:1 mode and then shutdown the application on one of the nodes while another node is starting the metavar exchange in CHM activation.

The code is modified to ensure that application is not going down before processing the metavar messages from other nodes.

Workaround: None.

Portfix SBX-111469: There are some problem causing MRFPs cannot recover from a restart

Impact: The standby SBC gets the already allocated metavars of an active node while coming up in an 'active' role.

Root Cause: The standby SBC is not checking for the condition to see if its the only node running in the redundancy group before allocating the last exchanged metavars of active node.

Steps to Replicate: Bring up the N:1, trigger a switchover so that assigned standby becomes active and while assigned active node is coming up in 'standby' role. Restart assigned standby and ensure the proper metavars are allocated to assigned standby node while its coming up in 'active' role.

The code is modified to check for the already allocated metavars before allocating the metavars to self node while coming up in the 'active' role.

Workaround: Restart assigned standby node if already allocated metavars are allotted to this node.

Portfix SBX-111310: Standby OAM is rebooting after orchestration.

Impact: The Standby OAM is restarting after a launch.

Root Cause: The Standby OAM is sending the serf event for 'startingStandby' with an older timestamp (which is fetched even before standby OAM is ready to come up) that results in discarding of the serf event by the active OAM node as its prior to member-join event for standby OAM node.

Steps to Replicate: Bring up both the active and standby OAM at the same time and then while standby OAM is coming up, disconnect and connect the HA port on active OAM to trigger member-failed and member-join events.

While sending the 'startingStandby' serf event, get the current timestamp so that its not discarded by the active OAM node.

Workaround: Restart the standby OAM application so that the startingStandby event is generated again with a current timestamp.

PortFix SBX-109811: The SBC uses port number RTP+1 for RTCP instead of the learned RTCP port number if the RTCP NAT learning completes before RTP NAT learning.

Impact: The SBC sends the RTCP packets to a destination port number RTP+1 for the RTCP instead of the learned RTCP port number if the RTCP NAT learning completes before RTP NAT learning.

Root Cause: The SBC overwrites the learned RTCP port number with the RTP+1 port number if the RTCP is learned before RTP.

Steps to Replicate: Send RTCP packets first then RTP packets to the SBC. After call is connected, verify the RTCP port number, if RTCP is learned before RTP.

The code is modified to use correct RTCP learned port when RTCP learning occurs before RTP, instead of comparing the RTP and RTCP addresses from callLeg structure.

Workaround: No workaround.

PortFix SBX-89177: A call is torn down upon a Hold (OA expiry).

Impact: A call flow involving an early media, tone and late Crankback results in a call tear down by the SBC while processing the HOLD INVITE.

Root Cause: While processing a 200 OK for INVITE, one of the call processing module ends up setting wrong App-Context-Id as active. Later, while processing a HOLD INVITE, this causes a failure since this module is not working on the latest App-Context-Id.

Steps to Replicate: Call Scenario:

1. Party A calls Party B through the SBC.
2. Party B sends 183 with SDP resulting in media cut-through at the SBC.
3. Later B sends 480.
4. The SBC is configured for the Crankback and as a result sends INVITE to Party C.
5. Party C sends 183 with SDP.
6. Party C sends 180 without SDP resulting in tone.
7. Party C sends 2xx with SDP for INVITE.
8. Party C sends HOLD INVITE.

Expected behavior: The SBC successfully process the HOLD INVITE.

Actual Behavior (without a fix): The SBC tears down the call while processing HOLD INVITE.

The code is modified to have the correct app Context Id as active while processing 200 OK for INVITE.

Workaround: Disable the Tones at the SBC.

PortFix SBX-110948: Load configuration overwrites the local processor index values.

Impact: When we perform a load config operation on a 1:1 HA pair (SWe/Cloud), it ends up overwriting the local processor index estimates with the ones that are present in the config dump.

This leads to two issues:

1. An incorrect index estimates being populated in the DB.
2. There are discrepancy in estimations between active and standby instances.

Root Cause: The load config operation results in overriding the local processor index estimates with the ones that are present in the config dump. This results in standby consuming incorrect indices present in the DB thereby causing the discrepancy in estimates of the active and standby instances.

The DB should always reflect the estimated processor indices calculated by the active instance in 1:1 HA pair.

Steps to Replicate: On 1:1 HA SWe/Cloud instances, perform the following steps:

1. Run the following command as root user on the active and standby instances. The following command should give same output on the active and standby instances.
   cat /opt/sonus/conf/swe/capacityEstimates/.index.txt
2. Perform the load config operation.
3. Run the following command as root user on the active and standby instances. The following command will give different output on the active and standby instances.
   cat /opt/sonus/conf/swe/capacityEstimates/.index.txt
   
   With the fix, this issue will not be observed.

The code is modified to ensure that the DB reflects the estimated processor indices calculated by the active instance in 1:1 HA pair.

If the processor indices values stored in the DB does not match with the indices calculated by the standby, then the standby goes for a reboot. From the next boot on-wards, the standby uses the indices stored in the DB for the session estimations.

The previously mentioned procedure ensures that the indices and sessions estimates are same on the active and standby instances.

Workaround: Run the following commands as root user on the active and standby instances.

1. The rm -f /opt/sonus/conf/swe/capacityEstimates/.indexMarker
2. Run a reboot

The SBC VNF cannot get to Ready after a VNFM Migration.

Impact: The VNF does not get ready after a migration to the new VNFM.

Root Cause: When the new VNFM is trying to send the curl request to the VNF, the request is getting rejected. When the VNF gets deployed, it contains the VNFM data in its user data and it makes one VNFM IPs allowed list. This allowed list is not getting updated for the new VNFMs. So, when VNF migrates to the new VNFM, new VNFM IP is not present in the allowed list.

Steps to Replicate: 

1. Launch VNFM with https.
2. Deploy a VNF and scale out the VNFM. Migrate the VNF to the new VNFM.
3. The VNF state should be read and available.

The code is modified so whenever the new VNFM sends a request to VNF, it is accepted.

Workaround: 

1. 1. Reboot the setup.
2. 2. Add new VNFM IP in the /opt/sonus/conf/instanceLca.json before the application comes up.
3. 3. Check the state of the VNF on VNFM.

SBX-99253: Customer ECGI to CA Mapping Enhancement.

Impact: The SMM Switch operation is not working after an SBC reboot.

Root Cause: The switchIndex is not being set properly during a config restore.

Steps to Replicate: 

1. Configure SMM Rule consisting of switch operation.
2. Run the Call.
3. Perform an SBC reboot.
4. Run the call (SMM will not be executed).

The code is modified to set correct the SwitchIndex during a config restore.

Workaround: After a reboot, we can delete the SMM profile and configure again.

CDR Viewer Download all button not working

Impact: The CDR Viewer download all button was not working.

Root Cause: An issue was caused when we changed the CSP. Due to that change, the content was made more strict and blob type were not allowed in Firefox and IE, resulting in a download failure.

Steps to Replicate: 

1. Log in to the EMA.
2. Navigate to CDR Viewer.
3. Select the Sip Ladder and click Download All.
4. Download is successful in Firefox/IE/Chrome.

The code is modified to allow the same to further processing.

Workaround: No workaround.

PortFix SBX-110956: Introduced the Upsampler for EVS in a EVS(wb)<=> NB codec scenario.

Impact: The WB will not be used for EVS encoding and as a result, the channel aware mode cannot be enabled in an IDP NB scenario though negotiated through the SDP.

Root Cause: There is an absence of an Upsampler in the Upstream path for EVS when WB is negotiated in an IDP NB scenario.

Steps to Replicate: Run a EVS<=>G711 call with bw=wb; ch-aw-recv-5;br=5.9-13.2.

In this case, the EVS encoder produces WB packets with channel aware mode enabled.

The code is modified for the EVS when the WB is negotiated in an IDP NB scenario.

Workaround: None

PortFix SBX-109300: The SBC should not accept cmr byte from discarded packets.

Impact: The SBC was accepting the CMR from discarded packets.

Root Cause: The CMR was being processed before Packet Validation.

Steps to Replicate: Run and EVS<=>EVS call. Let the peer send 32Kbps packets having CMR for 24.4Kbps.

In this case, the 32Kbps packets are discarded as we do not support transcoding beyond 24.4Kbps for EVS. Also since the packets are discarded CMR for 24,4Kbps is also not honored.

The code is modified to validate the packet first followed by the CMR processing.

Workaround: None.

PortFix SBX-109304: The 13.2 wb cmr is not accepted when the SBC is operating in channel aware mode.

Impact: The channel aware mode once enabled will always be enabled when the EVS encoder operates at 13.2Kbps and bandwidth WB. The only way to disable channel aware mode is to use a bitrate other than 13.2Kbps.

Root Cause: The code to disable the channel aware mode if enabled on receiving on 13.2 WB CMR was absent.

Steps to Replicate: Run a EVS to G711 call with br=13.2, ch-aw-recv-5; bw=wb.
Stream a pcap having CMR for 13.2 WB from the peer.

The call starts with 13.2 Kbps with Channel Aware mode being enabled. On receiving the CMR, the channel aware mode will be disabled while the bitrate is still 13.2.

The code is modified to address the issue.

Workaround: None.

PortFix SBX-110439: The DSP rejects CMR requesting channel-aware if localCodecMode is set to a value other than 13.2.

Impact: The SBC was rejecting a Channel Aware Mode CMR when the current mode of operation of EVS encoder was not 13.2Kbps WB.

Root Cause: A conditional check of the current Bandwidth and current Bitrate to honor a Channel Aware Mode CMR.

Steps to Replicate: Run a EVS to AMRWB call with the following SDP:

br=13.2-24.4; bw= nb-wb; ch-aw-recv=0

End point sends a CMR for Channel Aware Mode.

In this case, the call starts with EVS encoder encoding packets as 24.4Kbps and on receiving the CMR, the rate changes to 13.2Kbps with Channel Aware mode enabled.

The code is modified to check if 13.2Kbps is a part of the activeCodecSet and whether WB is present in the bandwidth range negotiated to honor a Channel Aware Mode CMR.

Workaround: None.

PortFix SBX-110299 to 9.2 - MRFP does not active EVS partial redundancy mode during the call when the remote offers "ch-aw-recv=0"

Impact: When the ch-aw-recv=0 is negotiated through the SDP, a CMR request for the Channel Aware mode with offset 2 and the priority HIGH was not being processed.

Root Cause: The root cause of this issue was a wrong initialization of the channel aware mode offset and priority in the code.

Steps to Replicate: 

1. Run a EVS to G711 call with br=13.2; bw=wb;ch-aw-recv=0
2. Stream a pcap with CA CMR for priority HI and offset 2.

Prior to the fix the CMR is not processed. With the fix, the CMR is processed and also honored.

The code is modified to address the issue.

Workaround: None.

The CDR record for DSP insertion/rejection reason field in 9.2.2R000 test execution has marked the call as “Transcoding” instead of “Transcoding due to DTMF”.

Impact: The CDR DSP insertion reason is showing as trancoded instead of the DTMF for transcoded call with difference in DTMF parameters on the ingress and egress leg.

Root Cause: The wrong code is present and overwrites the DSP insertion reason as transcoded.

Steps to Replicate: Make a successful transcoded call with AMR codec on both sides and the
DTMF parameter must be different in both ingress and egress.

The code is modified to set the DSP insertion reason based on the answered call leg.

Workaround: None.

The system level baseUdpPort is allowed to set higher than the range set at the IPTG level from the CLI.

Impact: It is possible to set the system media mediaPortRange baseUdpPort value to a greater value than the value assigned to all the associated trunk groups, which is invalid.

Example: set addressContext <name> zone <name> sipTrunkGroup <name> media
mediaIpInterfaceGroupName <IPIG name>media mediaPortRange baseUdpPort 1030

set system media mediaPortRange baseUdpPort 1050

Root Cause: With the introduction of the SBC and realms, the validation code was incorrectly looking for realm data to validate the system level baseUdpPort against rather than the zone/address context/trunk group information that is used in SBC configurations.

Steps to Replicate: Try to run the configuration below and check that an error report is now generated.

commit Aborted: 'system media mediaPortRange': System base UDP port cannot be greater than mediaPortRange configured at a trunk group

The code is modified to correctly use the zone/address context/trunk group information to validate the baseUdpPort information on the SBC configuration.

Workaround: Manually check the values on the trunk group before assigning the system level value.

Invalid value for "45.20 Reason Header value Q850" in the ACT record.

Impact: The invalid values populated in the CDR sub field 20 in ATTEMPT record.

Root Cause: In the Signaling Application, during Call Failure the value passed to CAM module to populate in the CDR for the sub field bit 20 was out of Q850 range. This is due to not mapping the internal CPC cause code to the corresponding Q850 standard values.

Steps to Replicate: Disable the Reason Header Q850 flag, and make a call. Reject the call with 607, where SIPTOCPC profile mapped as 607-216 post call release check the CDR Sub field - 20.

The code is modified to take care of mapping to right Q850 values.

Workaround: None.

The SBC inccorectly interprets RTCP packets as RTP when using DLRBT.

Impact: The RBT (ring back tone) can be terminated early when the DLRBT (dynamic local ring back tone) is enabled and RTCP packets arrive with the B-party.

Root Cause: When the DLRBT functionality was initiated it was not informed that RTCP could be received. This resulted in RTCP packets being treated as RTP and caused the SBC to think RTP was learned. As soon as SDP was received in 183, the SBC triggered cut through and the RBT was stopped even though the call was not answered. This left the A-party with silence until the call was answered.

Steps to Replicate: Make a PSTN to MS Teams call with DLRBT enabled. Leave the call in ringing state with MS Teams for a long time and check that the ring tone is continually generated.

The code is modified to correctly identify RTCP packets and not use this as an indication to media being learned so that the ring tone continues to be played until real RTP packets arrive.

Workaround: If RTCP is not required on the egress leg then disable it. If RTCP is required there is no work around.

Export/import of the syslog configuration fails.

Impact: The user-config-import command fails if the customer has configured the SBC to send the Linux level logs to a remote syslog server through the platformRsyslog configuration.

Root Cause: The child configuration objects under the platformRsyslog configuration were not being applied in the correct order during the import, which led to the configuration validation logic thinking there was no syslog server configuration and failing.

Steps to Replicate: Apply syslog configuration under the platformRsyslog and check that it can be exported and imported without errors.

The code is modified to correctly define the configuration order for platformRsyslog configuration so there are no errors during the import.

Workaround: If you edit the syslogState line in the xml file and change it from enabled to disabled, the import will complete correctly. Then manually apply the CLI command to enable the syslogState once the rest of the configuration is imported.

<platformRsyslog>
<syslogState>disabled</syslogState>
<linuxLogs>
<platformAuditLog>enabled</platformAuditLog>
<consoleLog>enabled</consoleLog>
<sftpLog>enabled</sftpLog>
<kernLog>enabled</kernLog>
<userLog>enabled</userLog>
<daemonLog>enabled</daemonLog>
<authLog>enabled</authLog>
<syslogLog>enabled</syslogLog>
<ntpLog>enabled</ntpLog>
<cronLog>enabled</cronLog>
</linuxLogs>
<servers>
<serverNo>server1</serverNo>
<remoteHost>2607:f160:10:4043:ce:ff0:0:35</remoteHost>
<protocolType>udp</protocolType>
</servers>
</platformRsyslog>

PortFix SBX-110984: The EVS CMR not honored when in dtx period.

Impact: If the SBC receives a CMR request when it is operating in DTX mode, the CMR was not being processed.

Root Cause: The rate or bandwidth change received through a CMR was not put into effect after coming out of the no transmission period.

Steps to Replicate: Run an EVS<=>EVS call with dtx enabled. On the Ingress leg, send a valid CMR while there is no media being sent on the Egress Leg. Send a CMR on the ingress leg when it is in the "no transmission period"

Once the Ingress Leg comes out of the no tx period, the bitrate or the bandwidth as requested by the CMR should be put into effect

The code is modified to address the issue.

Workaround: None.

Customer Physical Server had a reboot failure.

Impact: In a SBC Redundancy Group if an instance contains an SM core that did not cause service outage, the instance can go down when a switchover occurs between two other instances of same RG.

Root Cause: The coreHandler/FacHandler process gets called to dump the core during a coredump on SBC if FAC feature is enabled. This process registered itself in sysIdList that is unintended and causes SM to crash while handling instance down event for other instances of same RG.

Steps to Replicate: Create a RG, run it for few days and wait for an SM core due to the NTP, which does not cause the SBC to go down. Perform a switchover in same RG between two other nodes. The current node should not go down with fix build.

The code is modified to ensure the FacHandler does not register in sysIdList.

Workaround: None.

PortFix SBX-110850: Alter CPU Affinity of Processes in 2vcpu Scenario

Impact: There are intermittent crashes of SWe_NP/SWe_UXPAD processes in 2 vcpu deployments.

Root Cause: The SWe_NP and SWe_UXPAD DPDK processes running on the same core caused memory corruption in mempools.

Steps to Replicate: 

1. Bring up a 2 vcpu VM.
2. Run a heavy transcode call load for long duration.

The code is modified to launch the SWe_NP and SWe_UXPAD processes on different cores.

Workaround: There is no workaround for this issue.

The SCM Process cores on the customer PSBCs PS40, PS41, and PS42.

Impact: A SCM Process coredump was observed because of Segmentation fault when a call was intercepted with PCSILI and media monitoring enabled.

Root Cause: The SCM Process dumped core while pushing request for splitter resources and media monitoring was enabled.

Steps to Replicate: 

1. Configure the SBC and PSX for basic audio call.
2. Configure the customer CLI on Egress Leg for basic call.
3. Enable Media Monitoring on Egress leg for basic call.
4. Enable the Tones on Ingress Leg.
5. Perform an Interception based on PCSI_LI.

The code is modified to avoid the coredump.

Workaround: None.

PortFix SBX-109211: There was a design gap for AMR around dynamic codec transcoding.

Impact: The SBC is configured with restricted AMR/AMR-WB codec in Egress Route PSP along with Transcode "If Different DTMF PT" enabled then consider Egress endpoint selects the first pass thru restricted AMR/AMRWB payload in Offer in its 200 O.K answer. In such a case, if the DTMF attribute of the payload doesn't match then the SBC will try to match the Answered AMR/AMR-WB codec with the transcode-able AMR/AMR-WB codec. In such a case the DTMF attribute will match but the mode-set match fails that causes a call failure.

Root Cause:The SBC is configured with restricted AMR/AMR-WB codec in Egress Route PSP along with Transcode "If Different DTMF PT" enabled then consider Egress endpoint selects the first pass thru restricted AMR/AMRWB payload in Offer in its 200 O.K answer. In such a case, if the DTMF attribute of the payload doesn't match then the SBC will try to match the Answered AMR/AMR-WB codec with the transcode-able AMR/AMR-WB codec. In such a case the DTMF attribute will match but the mode-set match fails that causes a call failure.

The partial matching of AMR/AMR-WB codec in a Offer Answer cycle was allowed early, and that was what caused the issue.

Steps to Replicate: The SBC receives an INVITE with (AMR PT=96 and mode-set=1 and 2833 DTMF PT=101).
Egress route is configured with AMR mode-set=0, 1, 2, 3, AMR PT=126 and DTMF PT=110
Egress route is configured to transcode for Difference in 2833 PT
This causes the SBC to send out INVITE to UAS as follows:
96 (AMR mode-set=1)
126 (AMR mode-set=0,1,2,3)
110 (DTMF PT)

The code is modified so if the SBC transcodes the call due to Different DTMF, the transcoding is allowed and first offered payload type is used.

Workaround: None.

PortFix SBX-110009: The values in callsEstimate.txt differ between the Active and Standby SWe.

Impact: The session capacity estimate values differ in active and standby VMs upon an LSWU upgrade of 1:1 the SWe HA pair.

Root Cause: The following sequence of events resulted in the issue:

1. The processor indices are re-calculated upon LSWU upgrade, which differs from the processor indices calculated by the SBC VM in the earlier version.
2. The session estimates are calculated based on these new processor indices.
3. Post upgrade, once the application comes up, the processor indices that are stored in DB(from the older version VM), are restored back to the /opt/sonus/conf/swe/capacityEstimates/.index.txt file.

As a result, the Active and Standby VMs obtained different session estimates upon upgrade.

Steps to Replicate: After subjecting the 1:1 SWe HA to LSWU upgrade, content of the following file in Active and Standby VMs would differ:
/opt/sonus/conf/swe/capacityEstimates/.callsEstimate.txt

The code is modified to retain the processor indices during the LSWU procedure.

Workaround: After completing the LSWU upgrade, once the active(VM-A) and standby(VM-B) VMs are in sync, following a set of operations would make sure that session capacity estimates are identical in Active(VM-A) and Standby(VM-B) VMs:

1. Reboot the Standby (VM-B) VM.
2. The standby (VM-B) VM comes up with correct session estimates. Wait for the completion of application sync.
3. Now, reboot the Active(VM-A) VM. This results in application failover. Application running on the Standby(VM-B) VM takes over the active role.
4. Active(VM-A) VM comes up with the correct session estimates and comes in sync with the application running on the Standby(VM-B) VM.

PortFix SBX-105890: On a disk failure, the correct failover is not triggered.

Impact: On an SBC that was running 6.2 code, the disks on the SBC got into a bad state and would not process read or write operations. Automatic attempts to reboot the SBC from the application code failed and manual intervention was required to recover.

Root Cause: Later software releases already had better handling for this sort of issue but the code was printing multiple logs as part of the automatic reboot process and it was suspected that these could have gotten hung and the system could not get to the reboot command.

Steps to Replicate: The issue was not reproducible.

The code is modified to remove the logs to avoid the potential or not getting to actual reboot command in the code.

Workaround: The SBC needs to be manually power cycled to recover.

Portfix SBX-109591: Reject the INVITE with 100Rel when TG flag rel100Support is disabled and E2E Prack is disable on that leg after PSX DIP.

Impact: The SBC does not tear down the call if the INVITE contains a Require: 100rel and the rel100Support flag is disabled on the ingress sipTrunkGroup, as per RFC3262.

Root Cause: When the rel100Support flag is disabled and INVITE contains Require: 100rel, the SBC was not rejecting the Invite with 420 Bad extension. This scenario was not handled.

Steps to Replicate: Set this flag:
set addressContext default zone ZONE_IAD sipTrunkGroup TG_IAD signaling rel100Support disabled

When the INVITE is received with Require: 100rel and endToEndPrack is disabled, the SBC should reject the call with a 420 Bad extension and the SBC should send header Unsupported:100rel toward the ingress.

The code is modified so that when rel100Support flag is disabled and endToEndPrack is disabled.

If the INVITE contains a Require: 100rel, the SBC will reject the INVITE with 420 Bad extension and the SBC will send header Unsupported :100rel toward the ingress.

Workaround: None.

PortFix SBX-108410: [ASAN]: sanitizer.CE_2N_Comp_ScmProcess_3.8866:==CE_2N_Comp_ScmProcess_3==8866==ERROR: AddressSanitizer: heap-use-after-free on address 0x6190001c77dd at pc 0x558bcc9ff877 bp 0x7fea305f4e00 sp 0x7fea305f45b0

Impact: The ASAN reported a "AddressSanitizer: heap-use-after-free" error when a Subscribe request had a NULL character in a quoted string.
ie: From: "00 test"<sip:user1@host>

Root Cause: Invalid access of the freed memory occurred. Accessing memory after it is freed can cause unexpected behavior that may result in coredumps.

Steps to Replicate: Run the codenomicon subscribe-notify suite.

The code is modified so the SBC now logs a parser error If the SBC receives NULL character in a quoted string.

Workaround: None.

The one way audio when forked leg is MS Teams without ICE/NAT.

Impact: For an incoming call that is forked to multiple destinations and DLRBT is enabled, there is a possibility of audio flowing only in one direction when the call gets established.

Root Cause: When a 180 ringing is first received from one of the forked destinations, this triggering the SBC to play ringback tone, but if the call subsequently receives 18x messages and RTP media (for media cut through) from a different destination, the SBC fails to activate the RTP media flow resources correctly at the ingress leg of the call because these resources are already activated for playing the ringback tone for the other fork.

Steps to Replicate: Set up
---------
The SBC configured with call forking such that call from ingress (A) to be forked to egress (B) and egress (C).
The DLRBT should be enabled on ingress and egress trunk toneAndAnnouncementProfiles.

Procedure
--------------

1. Initiate the call with an INVITE from A that should be forked to B and C.
2. From the egress endpoint B respond with 180 and then from egress endpoint C respond with 180.
3. From egress endpoint C respond with 183 with SDP.
4. Send the RTP media packets back from C to cause media CUTTHRU.
5. From egress endpoint C respond with 200 OK to connect the call.

Expected Results
-----------------------
1. The INVITE sent to B and C.
2/3. The SBC sends back 180 towards A and starts to play ringback tone towards A.
4/5. The call is established, the SBC sends CANCEL towards B, ringback tone should stop and media should flow in both directions between A and C.

Before the fix, media was not flowing from A to C.

The code is modified to re-activate the media resources after the call is answered on one of the forks and the remaining forks have been cleared.

Workaround: Not available, but the issue does not occur if the DLRBT is not enabled.

PortFix SBX-105609: Add a check for /boot parition free space in pre-checks.

Impact: There was an upgrade failure due to insufficient disk space on /boot partition.

Root Cause: There was an upgrade failed due to failure to copy the new boot images as part of upgrade due to insufficient disk space in /boot partition.

Steps to Replicate: Upgrade to the fix build and ensure upgrade is successful.

The code is modified as part of pre-checks to ensure a minimum of 40MB free disk space is available on /boot partition.

Workaround: None.

PortFix SBX-110292: A Registration structure update is needed to prevent hijacking/hacking of the user after being rejected with a 403 Forbidden error.

Impact: After a hack, the registers rejected with 403 Calls were not working.

Root Cause: Both a normal and hacker user had very similar Register messages, except that the hacker's Auth header did not include the correct username. Due to this username mismatch, the SBC rejected the hacker with a 403 message, set the Register into 'challenged' state and later deleted the message.

Steps to Replicate: There are 8 combinations of test cases. The major difference is in the username field Auth header.

Test1:

1. Proper registration: Reg->401→Reg (with Auth)→200
2. Hacker registration: Reg->401→Reg (with hacker Auth)→403
3. Proper user refresh
4. Hacker registration: Reg->401→Reg (with hacker Auth)→403

Check for all fields and expiration timer.

Test2:

1. Proper registration: Reg->401→Reg (with Auth)→200
2. Hacker registration: Reg->401→Reg (with hacker Auth)→403
3. Proper user refresh
4. Hacker registration: Reg (with hacker Auth)→403

Test3:

1. Proper registration: Reg->401→Reg (with Auth)→200
2. Hacker registration: Reg (with hacker Auth)→403
3. Proper user refresh
4. Hacker registration: Reg->401→Reg (with hacker Auth)→403

Test4:

1. Proper registration: Reg->401→Reg (with Auth)→200
2. Hacker registration: Reg (with hacker Auth)→403
3. Proper user refresh
4. Hacker registration: Reg (with hacker Auth)→403

Test5:

1. Proper registration: Reg (with Auth)→200
2. Hacker registration: Reg->401→Reg (with hacker Auth)→403
3. Proper user refresh
4. Hacker registration: Reg->401→Reg (with hacker Auth)→403

Check for all fields and expiration timer.

Test 6:

1. Proper registration: Reg (with Auth)→200
2. Hacker registration: Reg->401→Reg (with hacker Auth)→403
3. Proper user refresh
4. Hacker registration: Reg (with hacker Auth)→403

Test 7:

1. Proper registration: Reg (with Auth)→200
2. Hacker registration: Reg (with hacker Auth)→403
3. Proper user refresh
4. Hacker registration: Reg->401->Reg(with hacker Auth)→403

Test 8:

1. Proper registration: Reg (with Auth)→200
2. Hacker registration: Reg (with hacker Auth)→403
3. Proper user refresh
4. Hacker registration: Reg (with hacker Auth)→403

The code is modified so when the SBC rejects the hack because of a username mismatch in the Auth header, it reverts back to previous details and state (that is complete).

Workaround: None.

PortFix SBX-103963: Both SBCs restarted at the same time and both mounted drbd0.

Impact: Both SBCs restarted at the same time and both mounted drbd0.

Root Cause: The PRS Process was not updating the syncStatus flag value, due to which the the standby was also rebooting thinking the sync is not complete yet.

Steps to Replicate: When both the the nodes are up and running, restart the standby. And while the standby is coming up, run a switchover from active CLI. The switchover should be successful.

The code is modified to use SMA API to check syncStatus instead of PRS and CHM local syncStatus flags.

Workaround: None.

PortFix SBX-110842: A switchover on the SBC 5000.

Impact: The SAM process core dumped when the SIP code was attempting to lookup a internal SIP Registration Control Block.

Root Cause: The core was the result of SIP code attempting to using a invalid index when accessing an array element for the purposes of finding a internal SIP Registration Control Block.

Steps to Replicate: We have been unable to reproduce this issue. The root cause was found by code inspection and has been corrected.

The code is modified to prevent the use of an invalid index when accessing an array element.

Workaround: There is no known workaround.

There are call trace logging issues while all Call Trace filters are disabled.

Impact: Messages traced at level 4 for sageTracing may erroneously have FILTER=<name> in the trace header. For the sageTracing, the filter name should be blank.

With the sageTracing enabled, 70% of the received INVITES are traced at the level 4 and 0.5% of calls have all their messages traced at level 4. The sageTracing is tracing collected for the Strategic Analysis Gap Execution.

Root Cause: The filter name messages traced for sageTracing should be blank but instead uses the 64th entry in the filter names table.

Steps to Replicate: 

1. The sageTracing is enabled.
2. Create at least 64 call trace filters and enable them.
3. Delete the trace filters.
4. Send INVITE messages.

The code is modified to filter the name blank.

Workaround: None.

Portfix SBX-106871: The SBC application times out while checking a callDetailStatus.

Impact: The CLI 'show table global callCountStatus' timed out after a call setup failure due to the codec license not being available.

Root Cause: The call setup failure in early stage caused an internal out of sync of the call resource, that triggers a handler of 'show table global callCountStatus' timeout when accessing the call resource.

Steps to Replicate: Create a call in the SBC with no available codec license of the call.

The code is modified to clear the call resources in all related internal modules upon call failure in early stage to address the issue.

Workaround: None.

Portfix SBX-100881: The SBC sends a release to the H323 side.

Impact: The SBC is sending a call release to the H323 side.

Root Cause: During the codec selection on H323 side, the SBC ran into some offer-answer collision due to ACK SDP from SIP side. As a result, the SBC is terminating the call.

Steps to Replicate: 

1. Make a call from SIP to H323 and the call gets connected.
2. Send a late media Re-INV from SIP EP to the SBC.
3. The SBC sends 200 OK with SDP to SIP EP.
4. SIP EP sends ACK with SDP to the SBC.
5. The SBC sends a FACILITY to H323 EP.
6. The SBC terminates the call.

The code is modified so that the codec selection on the H323 side occurs independently during a modify offer-answer is in progress.

Workaround: None

PortFix SBX-105657: The Customer SBC Upgrade failed.

Impact: The LSWU upgrade failed from 7.2.2R0 to 8.2.3R0.

Root Cause: The /tmp partition was running out of disk space and the restoration failed during an upgrade.

Steps to Replicate: Run a LSWU upgrade from 7.2.2R0 to 9.2.2.

The code is modified to ensure there is enough space available.

Workaround: Free up some space in /tmp directory and re-run the upgrade.

PortFix SBX-110247: The "sonusSbxNodePolicerMinorAlarmNotification" alarm is generated by the SBC.

Impact: Packets arriving on a unallocated port do not get marked as rogue media.

Root Cause: The issue occurs only on ports that was used and disabled. Packets arriving on this port get marked as grace-media packets. This is because a very high value of grace period is being incorrectly programmed for the udp port instead of the default 4 seconds. The high value is because of endianness.

Steps to Replicate: 

1. Reboot the VM.
2. Make a single normal call and wait for the call to end. Note down RTP UDP port of the SBC used in the call.
3. After the call has ended, stream UDP packets to the same RTP UDP port previously used in the SBC call. We will not see any unallocated port rogue media entry for the stream.
4. Stream the UDP packets to a UDP port that has not been previously used in the SBC call. We will see rogue media entry for the stream.

The code is modified to perform the appropriate endianness translation on the affected fields to arrive at the correct value.

Workaround: No workaround. This does not affect normal media processing, or associated statistics.

PortFix SBX-110323: A SWe media Issue occurred after an active reboot

Impact: Media issue after switchover.

Root Cause: Before a switchover, the loopback call was set up with 2:xx:xx:x:x:x (vbsc1's pkt port mac address) for both src and dst MAC address. After switchover, from packet capture, we found that the SRC MAC address= 2:xx:xx:x:x:xx (vsbc2's pkt port mac address). As a result, using the srcMac and dstMac caused a media issue.

Steps to Replicate: Use the following configurations to test:

• Set up loopback and normal calls for both SBC HW and SWe platforms.
• Do port and SBC switchovers.
• Use single and alternate media IP addresses on the loopback ports.

The code is modified to use loopback flag mirrored to standby BRM, and overwrite both srcMac and dstMac if loopback flag is set to true. This is done only on SWe when enabling or modifying the RID.

Workaround: No workaround.

PortFix SBX-109200: Missing the CDR for Teams call transfer scenario.

Impact: For an MS Teams blind transfer scenario with tonesAsAnnouncements enabled, where the MS Teams user A establishes a call to PSTN user B, then MS Teams user A establishes a call to user C and then invokes a blind transfer to establish the call between B and C. When the call is subsequently cleared one of the CDR STOP records for the call is not generated.

Root Cause: When the tonesAsAnnouncements are enabled, after a blind transfer is initiated from user A, the subsequent call clearing logic is not correctly deactivating the internal announcement resources causing the call to not fully clear internally which results in STOP record not being generated.

Steps to Replicate: With the SBC configured for MS Teams having DLRBT with announcementBasedTones enabled.

Procedure
--------------

1. Establish a call from MS Teams User A to PSTN user B.
2. Establish a call from MS Teams user A to PSTN user C.
3. Initiate a blind transfer from MS Team user A, between user B and user C.
4. Clear the call and check CDR records.

Expected Results
-----------------------
Calls, transfers, and announcements should be successful.

The SBC should generate START and STOP CDR records for A-B and A-C call.

The code is modified to correctly deallocate announcement resources for this call scenario so subsequent call clear can complete successfully and generate the correct CDR stop records.

Workaround: If tonesAsAnnouncements are disabled the call scenario works as expected.

Portfix SBX-107976: Disable the FAC feature in M-SBC, and SLB.

Impact: Non-SIP SBC personalities should not have FAC feature enabled.

Root Cause: Earlier, the SBC did not have personality check when setting core pattern.

Steps to Replicate: Verify the user defined core pattern is not set in /proc/sys/kernel/core_pattern when instances are spawned with the personalities msbc,slb,mrfp irrespective of facState.
%set system faultAvalancheControl facState <enabled/disabled>

User Defined Pattern:

cat /proc/sys/kernel/core_pattern
|/opt/sonus/sbx/bin/CoreHandler -f /var/log/sonus/sbx/coredump/core.1.%e_%p.%t -t %i -p %p

Default Core Pattern:
cat /proc/sys/kernel/core_pattern
/var/log/sonus/sbx/coredump/core.1.%e_%p.%t

The code is modified to not enable the user defined core pattern for personality type M-SBC, and SLB.

Workaround: Disable the FAC Feature.
%set system faultAvalancheControl facState disabled

PortFix SBX-107583: There is one way audio on hairpinned calls.

Impact: A one way audio problem is seen in a AMR-AMR transcoded call with DTMF interworking when "Different2833PTType" is enabled. This problem is visible for all HD codecs (AMR, AMR-WB, EVS, SILK) that uses dynamic payload in SDP offer.

Root Cause: A one way audio problem is seen in a AMR-AMR transcoded call with DTMF interworking when "Different2833PTType" is enabled. This occurs because SBC incorrectly configures the DSP channel with an incorrect payload type, not the payload type which was negotiated during the Offer Answer exchange on the Egress leg.

Steps to Replicate: 

1. Enable "Different2833PTType" and TransCode conditional in PSP and set Preferred DTMF payload to 102 on both the PSP.
2. Send AMR payload 96,AMR-WB with DTMF 110 (8k) from Ingress.
3. Ensure Egress answer sends AMR 96 and DTMF 102 in answer.
4. Ensure that the call is transcoded and there is no way audio problem.

The code is modified so the SBC configures the DSP with pass thru payload type.

Workaround: A code fix has been made.

Portfix SBX-107954: The SBC CPXA coredump after EVS+T140 MRFP call.

Impact: Occasionally, when the show global callDetailStatus command is executed in the CLI, the CPX process coredumps.

Root Cause: If information is not available for a particular call, the data returned did not have a valid type for the Ip addresses returned. This caused a timeout out in confd because the CPX did not return information for the global callDetailStatus.

Steps to Replicate: The steps cannot be reproduced.

1. The code is modified to return valid information for a call when information is not available.
2. The code is also modified to send a response to the CPX process when the application does not return proper data so that the CPX process will not crash.

Workaround: Do not run the global callDetailStatus command.

PortFix SBX-108557: The SBC continuously core dumps for SCM process since the upgrade to V09.02.01R000.

Impact: ScmProcess may coredump due to memory corruption.

Root Cause: There is code that is using an invalid pointer when writing to a buffer. This code was only added recently in 9.2.1R0.

Steps to Replicate: This problem is triggered by the receipt of an invalid PDU and/or an SMM rule to reject the incoming Invite and an early ATTEMPT record was attempted to be written. The issue is random and depends on what info is NOT available when trying to write the accounting record, therefore the issue may not reproduce all the time.

The code that uses in invalid buffer pointer (which was added recently) has been removed.

Workaround: If there is SMM rule (ignore/reject the Invite), then the SMM rule needs to be disabled until patch is applied.

PortFix SBX-108126: Observed a SAM Process crash in another active node during SIPFE crash testing.

Impact: The SAM Process coredumps due to a buffer overflow.

Root Cause: The key:value length of incoming Message is max 255 bytes. But in the yang spec, the buffer length was defined as 23 bytes. As a result of this issue, the display was truncated and also crashing due to buffer overflow.

Steps to Replicate:

1. Configure the testbed for the fault avalanche testing.

2. Trigger the crash by sending fac-sipfe-0 in the From header, where From(calling Party) is larger than 23 bytes.

Expectation:

1. The current active should crash the SAM Process.
2. Standby will take over as active and blocks the SIP message that triggered crash.

The code is modified to handle 255 bytes string, so the buffer overflow does not happen and display is not truncated.

Workaround: None.

PortFix SBX-108388: The SBC set "user=phone" on Dummy History-Info header.

Impact: When interworking between diversion header and history info header if the diversion count is more than 2 then the SBC generates dummy history info headers. These dummy history info headers contain user=phone which some end points reject.

Example of a dummy history info header.

History-Info: sip:unknown@unknown.invalid;cause=302;user=phone;index=1.1;mp=1

Root Cause: The code was mistakenly adding the user=phone attribute even when no phone number was present in the dummy history info header.

Steps to Replicate: Configure the SBC to support interworking from diversion header with a count of 5 to history info headers and check that the dummy history info headers do not include user=phone.

The code is modified not to include user=phone in the dummy history info headers.

Workaround: Use SMM to remove the user=phone attribute from the dummy history info headers.

PortFix SBX-109464: Using a leadership algorithm workaround for an old openclovis issue can cause a core dump.

Impact: Thesafplus_gms process crashes when coming out of split brain.

Root Cause: There was incorrect/inconsistent data results in the code asserting.

Steps to Replicate: This problem is not easily reproducible and is caused by HA link instability/flapping.

The code is modified so that the data is consistent.

Workaround: The HA link stability.

Portfix SBX-109224: AWS: The upgrade is failing with an error message " instance is not reachable " even though the upgrade status is "success :true"

Impact: The AWS upgrade was failing due to instance non-reachability. In actuality, the VNFC was up and running but the VNFR showed VNFC status non-reachable.

Root Cause: The VNFR-VNFC communicates using a ZMQ and VNFR updates the VNFC health check using the same ZMQ thread mechanism. There is a Zmqclient and ZmqServer. After an undetermined point in time, the VNFC(ZmqClient) is waiting for an infinite time to get a response from the VNFR(ZmqServer). This is leading to blocking state.

Steps to Replicate: Run an upgrade/revert operations on 9.2.2/10.0.0.

The code is modified so the ZMQ communicates in non-blocking mode.

Workaround: For the workaround, the user needs to run the pre check command from vnfr_intf before upgrade/revert operations and needs to take action provided as a part of output table.

Portfix SBX-109922: The buffer used while printing PSP's was running out. As a result, this is an error.

Impact: The buffer used while printing PSP's was running out because of large PSP. As a result, this is an error.

Root Cause: The buffer used while printing PSP's was running out. As a result, this is an error.

Steps to Replicate: 

1. Run the suite with the RTCP Interworking Enhancements.
2. The warning above in the Opensaf logs should not come.

The code is modified to accommodate bigger strings.

Workaround: None.

Portfix SBX-109336: The SBC 5110: BIOS not updated to 2.07 post-upgrade as indicated in Release Notes.

Impact: The BIOS is not upgrading part of the SBC application upgrade.

Root Cause: The flashroom utility is using /dev and /proc file system to upgrade BIOS. these file systems are un-mounted part of grub2 config upgrade.

Steps to Replicate: Upgrade the SBC app from 7.x (BIOS=2.6) to >= 8.1.x.(BIOS = 2.7). The BIOS should upgrade part of an application upgrade.

Mount /dev and /proc file system before calling BIOS upgrade

Workaround: 

1. Stop SBC App
   1. #sbxstop
2. Go to /opt/sonus/bios-update/
   1. #cd /opt/sonus/bios-update/
3. Run the script to update BIOS.
   1. # ./updateBIOS.sh bios5X00_v2.7.0-R0.rom

This scripts will take few minutes to update BIOS, after its done reboot host. In next boot it will boot up with new BIOS.

Portfix SBX-108127: 9.1r3 to 10.0 LSWU failed with the reason "Failed_to_install_or_configure_database"

Impact: The LSWU failed with the reason "Failed_to_install_or_configure_database".

Root Cause: Ownership for the files in /var/log/postgresql is getting changed at the time of the upgrade.

Steps to Replicate: Perform an upgrade from any version to 9.x

The code is modified to restore ownership of the log files in /var/log/postgresql to postgress.

Workaround: None.

PortFix SBX-108237: Performance: Observed SAM and SCM process coredump for FAC enabled execution on SBC 5400

Impact: The SCM experiences a core dump when the Invite gets a transaction timeout and the 200 Ok for Invite is received at the same time.

Root Cause: In the problem scenario, the SBC is trying to send ACK for the 200 OK, the SCM cores when creating a SIP Transaction for ACK Message.

Steps to Replicate: 

1. Run a Basic Invite call flow.
2. Do not answer for Invite on the UAS side.
3. After 32 seconds send 200 OK for Invite from the UAC side.

A race condition might occur and result in coredump.

The code is modified not to send ACK in this scenario as the CseqType and CseNumber are unknown and 0 respectively.

Workaround: Not Applicable.

PortFix SBX-107690: There are SBC call failures observed on T140 load with various MAJOR logs in DBG.

Impact: A call fails due to RID Enable errors. (where RID = receiver ID and is mapped to an allocated resource).
ThenDBG log shows many BrmAsnycnCmdErrHdlr logs with cmd 0x30 (RID Enable):
MAJOR .BRM: *BrmAsyncCmdErrHdlr: ERROR NpMediaIntf cmd 0x30 gcid 0x2128915e

Root Cause: When the RTCP Generation is disabled, the RTCP RID for the call is expected to be disabled by Network Processor (NP).

With the introduction of SBX-86241 "Streaming RTCP packets to Protect Server", we now have a case where RTCP Generation is enabled to stream RTCP packets to Protect Server but RTCP packets are not generated for the call and RTCP RID for the call is not enabled.

When the RTCP Generation is disabled, the NP uses rtcpMode=RTCP Terminate to indicate that the RTCP RID needs to be disabled.

This is incorrect since rtcpMode=RTCP Relay Monitor also has the RTCP RID enabled and NP is expected to disable the RTCP RID.

If a call has the rtcpMode=RTCP Relay Monitor and RTCP Generation is disabled, we leak this particular RTCP RID resource.

The next time, we try to allocate this particular RTCP RID, NP returns an error indicating RTCP RID is already allocated.

Steps to Replicate: 

1. Enable Packet Service Profile //Resources/profiles/media/packetServiceProfile/rtcpOptions/generateRtcpForT140IfNotReceivedFromOtherLeg.
2. On SBC, set //System/media/mediaRtcpControl/t140RtcpMonitorInterval to 20 seconds.
3. Start T140 call but do not send RTCP packet. Terminate call in 10 seconds.
4. If you keep making this type of call, you will eventually see the Enable RID error. DBG log will have the BrmAsynCmdErrHdlr entry.

The code is modified so the Bus Resource Manager (BRM) knows whether RTCP RID is allocated and needs to be disabled by NP.
The code is also modified so that BRM indicates RTCP RID needs to be disabled or not.

When NP receives the command, it uses this new parameter to decide if RTCP RID should be disabled or not.

Workaround: Disable RTCP and disable RTCP termination.

Portfix SBX-109384: The 8.2.4F001 SBC 5400 Global level SMM stopped to work.

Impact: The Global SMM function stopped working post upgrade and similar issue was observed after multiple switchovers.

Root Cause: The Global SMM configurations were not being restored.

Steps to Replicate: 

1. Attach an SMM at global level (for 200ok of pathcheck options).
2. Perform multiple switchovers or LSWU to another build.
3. Check if the SMM is being applied.

Restore the Global SMM configurations during multiple switchovers to address the issue.

Workaround: Delete the profile set at global level and configure it again.

PortFix SBX-107430: URI parameter setting order of R-URI does not meet RFC 3966 requirements.

Impact: In the request URI, the isub parameter was after the NPDI parameter, which does not meet the RFC 3966 requirement. (ie. npdi;isub=1111)

Root Cause: In the code (sipsgCallProcEngine.c), the isub parameter was getting populated after NPDI and other user parameter.

Steps to Replicate: SipSgMapIsdnSubAddrToSipParam this function is responsible for populating isub parameter to request uri, this moved prior to all other parameter populating functions.

The code is modified so the isub parameter populating function is moved before all other parameter populating functions.

Workaround: NA

PortFix SBX-108270: The S-SBC sends a DNS SRV before completing all NAPTR query.

Impact: The S-SBC sends DNS SRV before completing all NAPTR query.

Root Cause: A DNS agent request timeout occurs after 10 seconds. Because the DNS agent sends the DNS lookup (NAPTR) request to DNS Client process, the DNS client process then queries to an external DNS server for NAPTR records. The DNS agent waits until 10 seconds then timeout's for each DNS lookup. As a result, the DNS request is failed with the first DNS server and query is still in process for a second DNS server.

Steps to Replicate: Configure 2 DNS server's

1. Configure DNS global configuration to:
   admin@SBX136% show global servers DNS global
   retransmissionCount 14;
   retransmissionTimer 500;
   set addressContext default dnsGroup DNSGrp1 negativeDnsCacheSupport disabled
2. Configure dnslookupTimeoutTimer to 10 sec.
3. Stop the DNS server.
4. Make a call.

Observation:

1. DNS should retransmit DNS NAPTR query to DNS server1 and after should move to DNS server2.
2. After 10 sec, NAPTR query shall stop.
3. The SBC should send SRV/A query to DNS Server2.

The code is modified to:

1. Add new configuration timer flag "dnslookupTimeoutTimer" for DNS lookup.
2. Once the lookup timer expires, the DNS client will stop sending same query to DNS server (Ex: if NAPTR query is re-transmitted towards the DNS server, after timer expire. NAPTR query will be stopped).

Workaround: None.

PortFix SBX-107517: Import configuration results in two different final status depending where you look at it.

Impact: The postgres DB restore fails that leads to provisioning issues such as:
The "show configuration" does not display a complete configuration related to policy entities like sipTrunkGroup, IpPeer etc.
The "delete" operation fails for sipTrunkGroup and other policy related entities. The exportConfig does not contain policy related entitles.

Root Cause: The PostgressDB restore is failing due to restricted permission.

Steps to Replicate: Set up required: 1to1 S-SBC virtual platform and register it within Direct-Single type of cluster on EMS.

1. Configure 2 TG and saveAndActivate revision (e.g. revision 2).
2. Configure couple of more CLIs and save AndActivate (e.g revision 3).
3. Restore revision 2 either from CLI or EMS.
4. After restore is successful. try to delete a TG. Observed Application failure.

After implementing the fix, use the 4 steps and as a result, the delete was successful.

The code is modified to permit an appropriate permission to postgres user while restoring the pg_policy dump.

Workaround: Use the following steps:
#cd /opt/sonus/sbx/scripts

Update the oam-config.sh on active and standby the SBC as following:

replace
$CHMOD -R 660 $extractDir/

with

$CHMOD -R 665 $extractDir/
accessRights=`$STAT -c %a $SONUS_TMP_DIR`
$CHMOD 775 $SONUS_TMP_DIR

find $RM -rf $extractDir and add following line below this

$CHMOD $accessRights $SONUS_TMP_DIR

Restore a revision that has pg_policy_xx.dump with correct Trunkgroup, ipPeer etc. entries.

Portfix SBX-110354: After upgrading from V08.02 or V09.02 to 10.00.00R000 successfully, a REVERT from 10.00.00R000 to V08.02.04R000 is failing.

Impact: Reverting from the higher software version to lower software version fails.

Root Cause: The OAM does not upload restored revision against lower software version (ie. restore of config tar ball that was created on lower software version before upgrade). After following the MOP for downgrade when OAM comes up, it downloads the last revision uploaded on EMS (in this case config revision for higher software version). As a result, it fails to start the service because of a version mismatch.

Steps to Replicate: 

1. Spawn OAM + S-SBC on V08.02.04R000.
2. Create multiple revision (e.g revision 1, 2, 3, 4).
3. Upgrade to software version V10.00.00R000.
   The revision 5 gets created from revision 4,
   Create revision 6, 7, 8.
4. Restore to revision 4 -> revision 9 will get created and uploaded to EMS.
5. Downgrade to revision V08.02.04R000.

Expected O/P:

The OAM should come up with revision 10 created from revision 9.

The code is modified to upload a configuration when a revision that is created on a lower software version is restored. And after a  downgrade, it picks the revision with software version where the downgrade is done.

Workaround: None.

Portfix SBX-106475: The Cloud SBC instances are not coming up after a fresh installation or rebuild and the MGMT I Pis unreachable in BLR-PC3.

Impact: The SBC instances are not coming up after a fresh installation or rebuild and the mgmt ip is unreachable when using the X710 sriov vfs for pkt interfaces.

Root Cause: The X710 driver reset has been known to experience a delay in completing, thus causing a cloud-init service failure. For example, after the OS boot (as part of renaming the interfaces by sonusdev), the X710 driver is restarted. When the X710 does not immediately reset, the next cloud-init service fails, and the system is not in proper state.

Steps to Replicate: The SBC instances should come up properly after fresh installation or rebuild and mgmt ip should be reachable with all supported sriov vfs for pkt interfaces.

The code is modified so after the renaming is done and the network driver is restarted, wait for some seconds to check if all the nic ports are up and running in sonusdev itself, before exiting the service. This blocks all other dependent service from starting. Once the nic port are up, proceed with the system bring up. With these code changes, the nic does not come up then, we log it as critical log and expect the user to check on it.

Workaround: None.

Portfix SBX-109597: Observed SCM process crash on the newly active S-SBC during SWO testing.

Impact: While the SBC is running in sensitive mode, observed the SCM Process core during a switchover testing under the 1000 cps call load.

Root Cause: As part of switchover, all the connected calls are rebuilt on a newly active SBC. But, there may be a chance that the SBC can hold few stale entries related to active calls in the system, which will be cleaned up after some time. If any of these stale entries receive an unexpected events, that leads to a core dump.

Steps to Replicate: 

1. Run 1000 cps load with 120 CHT.
2. Perform a switchover by killing the PRS Process.

The code is modified to identify these events for the stale entries and ignore them to avoid a coredump when the SBC running in the sensitive mode.

Workaround: None.

PortFix SBX-108225: Observed core files on fresh installed VMware SWe.

Impact: The SSREG and SLWRESD processes a coredump on VMware SWe due to the time being set in past.

Root Cause: When the SBC is installed using ISO and then app is installed, the ntp sync is occurring while the application is running and in case the time is set in the past, it causes SSREQ and SLWRESD processes to core dump.

Steps to Replicate: Launch with the NTP IP other than a default IP, check if a coredump occurs due to time being set in the past.

The code is modified to update the /etc/ntp.conf before the SBC comes up so the runntpdate can access the NTP server IP.

Workaround: No workaround.

Portfix SBX-108219: Observed a SAM Process coredump in SLB for 1000 cps Peering Call load.

Impact: In a load run, there was a core being observed in the SLBDEBUG call when it hits the default states as part of the SBC message processing.

Root Cause: In the SLB debug call, there was new line () being passed as part of a string.

Steps to Replicate: Not reproducible. The issue seen once when running.
Run 1000 cps INVITE call load with PRACK enabled.

The code is modified so the new line () is not part of the debug string.

Workaround: There is no workaround identified for this issue.

PortFix SBX-110642: A coredump occurs after a blind transfer call.

Impact: The DNS process is coring when TEAMS blind transfer call is made.

Root Cause: The NULLError check was missing before accessing DNS Group pointer in DNS module. This occurs when DNS Group is not attach to Zone. So get function for the DNS group pointer returns NULL.

Steps to Replicate: PSTN to TEAMS

TEAMS blind transfer to PSTN2

Expected Result:
There should not be a coredump during the BT.

The code is modified so now the DNS group is fetched after setting the current DNS group to default DNS group, this is case when the DNS group IP not provided.

Workaround: No workaround.

Portfix SBX-108612: An SCM Process core dump occurred when INVITE with message size of 31700 bytes is sent

Impact: Receipt of SIP messages with Content-Type: multipart/mixed and more than 10 content entries cause coredump, if a message manipulation rule is active with criterion on message body.

Root Cause: Missing handling for unexpected message received.

Steps to Replicate: 

1. Configure and apply a message manipulation rule with criterion messageBody.
2. Send in INVITE with Content-Type: multipart/mixed and more than 10 content entries.

The code is modified so that if more than 10 content entries are present, the message is not considered for SMM rule actions with criterion on message body.

Workaround: None.

Portfix SBX-109327: Some CDR Events in ACT files have timestamps that are not current.

Impact: Duration Field (+24 Duration of SIP Registration/Subscription Context) is not displayed for some register transaction.

Root Cause: Duration Field (+24 Duration of SIP Registration/Subscription Context) is only updated in case of de-register for Successful transaction.

Steps to Replicate: 

1. Configure SBC for Registration/Subscription.
2. Perform following Registration.
   A ------REGISTRATION----->SBC-----REGISTRATION---->B
   A<--------404--------------------SBC<-----------404---------------B
   A ------REGISTRATION----->SBC-----REGISTRATION---->B
   A<--------200OK--------------------SBC<-----------200OK---------------B
3. Check CDR for "Duration Field" ( +24 Duration of SIP Registration/Subscription Context)

The code is modified to display the duration of SIP Registration/Subscription context for every CDR event.

Workaround: None.

PortFix SBX-106760: Performance: While running IMS AKA Registrations on SWe KVM, cannot achieve 1000 subscribers with 30 RPS properly.

Impact: The SBC was not able to achieve the IMS AKA registrations even at a lower rate (30 registrations per second).

Root Cause: For one of the IPsec features, the XRM code was modified to perform route lookup and next hop destination MAC resolution during IPsec SA setup. This was causing a delay in setting up IPsec SAs during load causing timeouts during the IMS AKA registration.

Steps to Replicate: Run the IMS AKA Registration load at 30rps.

The code is modified to not invoke the route loopup and destination MAC resolution code that is causing delay in setting up IPsec SA, which addresses the issue.

Workaround: None.

Portfix SBX-110066: Observed a SAM Process core dump in SLB while testing Performance with SLB (TLS Peering Calls) - One time Occurrence.

Impact: A SAM Process core was observed on a shutdown.

Root Cause: A race condition in the process shutdown code occurred allowing access to an invalid pointer, and causing a core dump during the shutdown of the SAM Process.

Steps to Replicate: Restart the SBC and look for a SAM Process core.

The code is modified to avoid the race condition that led to the core.

Workaround: No workaround, but since this core is during shutdown, there is no impact to normal operations of the SBX.

PortFix SBX-109953: Performance: Observed NP_WRK0 Core on Active Instance on OpenStack while using VIRTIO.

Impact: The SWe_NP can crash during a sbxrestart or sbxstop in the extra small memory profile.

Root Cause: There is corruption due to incorrect use of global variable in an extra small memory profile.

Steps to Replicate: 

1. Bring up the SBC in extra small memory profile.
2. Run a sbxrestart or sbxstop.

The code is modified to address the issue.

Workaround: None.

PortFix SBX-108310: Various link monitor issues on the SBC SWEs with port redundancy enabled.

Impact: On the VMware platform with Intel 82599 SR-IOV packet port VFs, the link does not get restored upon toggling the underlying physical link.

Root Cause: Because of VMware PF driver shortcoming, DPDK ixgbevf PMD code does not get link up indication from link speed status register for SR-IOV VFs from Intel 82599 card.

Steps to Replicate: 

1. Bring up port redundancy setup on VMware with SR-IOV 82599 VFs packet port.
2. Toggle the carrier link state on the NIC by cable pull or interface link toggle on host through the IP commands.
3. Observe the link status is not restored.

The code is modified to detect the physical link loss from the NACK status set on the hardware mailbox register, specifically for the VMware platform.

Workaround: No workaround. Only reboot clears the link state.

Portfix SBX-106858: The SBC clears a call on receipt of 200OK answer - CFNA call.

Impact: The SBC clears call on receipt of answer for a MRF transcoded call involving egress UPDATE followed by tone play.

Root Cause: During tone play, the SBC detaches the MRF resource from the call since tone is played by SBC. However, the media endpoint resource facing MRF stays with the call that results in failure later while assigning MRF resource back to the call during answer processing.

Steps to Replicate: The following events lead to failure in this call flow:

1. 183 with SDP from egress leads to cut-thru (MRF transcoded call).
2. Then, egress peer sends SIP UPDATE to the SBC.
3. Followed by 180 from egress which arms RTP monitoring at the SBC.
4. Followed by RTP failure notification after 2 seconds which starts Tone.
5. Followed by answer which results in failure.

Detach the media endpoint resource facing MRF from the call during tone play to address the issue.

Workaround: None.

Portfix SBX-107492: Observed a SCM Process core dump after 2 hours of load run with EVRCB to G711 transcode calls.

Impact: During call load, the SIPFE module was crashing while writing date to a mapped address.

Root Cause: The write to memory mapped address for files was causing a delay and leading to health check time outs causing core dumps

Steps to Replicate: Run a 100 cps SIP-SIP call load for about 2- 6hrs period of time.

The code is modified from file to Shared memory, which solved the problem

Workaround: None.

Portfix SBX-110184: Performance: While pumping Emergency call load on top of normal call load, HPC calls are getting rejected with 488 sip Error due to dsp overload even though resources were reserved on Openstack D-SBC HA.

Impact: The HPC calls are rejected with 488 even though the SBC has sufficient DSP resources reserved for such calls through the highPriorityCompressionReservation configuration.

Root Cause: In a D-SBC setup, when DSP allocation request is received, the T-SBC applies the call admission policy to know whether the call can be admitted or not before proceeding with DSP allocation. In this process, the T-SBC was not considering if the allocation request was for HPC call or normal call. As a result, the HPC calls were treated like normal call leading to rejections if this T-SBC is running under high load.

Steps to Replicate: 

1. Reserve DSPs for HPC calls.
2. Run a normal call transcoding at 200cps.
3. Run HPC transcoded calls at 5cps.
4. Ensure the HPC calls work as long as reserved DSP resources are not exhausted.

The code is modified to give preferential treatment to HPC calls during call admission policy at the T-SBC.

Workaround: No workaround.

A DEADLOCK was detected for sysID 85, task CHM.

Impact: Health check timeout resulted in deadlock for CHM and process crash to recover.

Root Cause: The CDB read call took longer than a health check interval resulting in CHM process being crashed to recover.

Steps to Replicate: The problem could not be reproduced.

The code is modified so the health check timeout is increased to 30 now from previous 10. The longer health check interval allows for unexpected cases where processing can take longer than expected especially on SWe systems.

Workaround: None.

The MGMT Port Status was showing the wrong status.

Impact: An Unconfigured Management port is shown as UP when there is no cable connected to the port.

Root Cause: The SBC did not keep track of management port status if the management port is not configured/used.

Steps to Replicate: 

1. Show the issue.
   1. Leave SBC 5400 mgt2 port unconnected.
   2. Display mgmtPortStatus - the unconnected mgt2 shown as admnEnabledPortUp. This is the issue.
      admin@YF01> show table system ethernetPort mgmtPortStatus
      CE PORT IF NEGOTIATED DUPLEX RX ACTUAL TX ACTUAL RX TX RX
      NAME NAME INDEX MAC ADDRESS SPEED LINK STATE MODE BANDWIDTH BANDWIDTH PACKETS PACKETS ER
      -------------------------------------------------------------------------------------------------------------------------
      YF01 mgt0 1 0:10:6b:3:c8:d speed1000Mbps admnEnabledPortUp full 232 0 45 3 0
      YF01 mgt1 2 0:10:6b:3:c8:e speed1000Mbps admnEnabledPortUp full 0 0 6 2 0
      YF01 mgt2 3 0:10:6b:3:c8:f unknown admnEnabledPortUp unknown 0 0 0 0 0
      YF01 mgt3 4 0:10:6b:3:c8:10 speed1000Mbps admnEnabledPortUp full 0 142 2 42 0
      [ok][2021-05-18 10:32:15]
2. Show the fix.
   1. Leave SBC 5400 mgt2 port unconnected.
   2. Display mgmtPortStatus - the unconnected mgt2 shown as admnEnabledPortDown (this is the fix).
      admin@YF01> show table system ethernetPort mgmtPortStatus
      CE PORT IF NEGOTIATED DUPLEX RX ACTUAL TX ACTUAL RX TX
      NAME NAME INDEX MAC ADDRESS SPEED LINK STATE MODE BANDWIDTH BANDWIDTH PACKETS PACKETS
      -------------------------------------------------------------------------------------------------------------------------
      YF01 mgt0 1 0:10:6b:3:c8:d speed1000Mbps admnEnabledPortUp full 248 0 271 6
      YF01 mgt1 2 0:10:6b:3:c8:e speed1000Mbps admnEnabledPortUp full 0 286 27 103
      YF01 mgt2 3 0:10:6b:3:c8:f unknown admnEnabledPortDown unknown 0 0 0 0
      YF01 mgt3 4 0:10:6b:3:c8:10 speed1000Mbps admnEnabledPortUp full 0 0 0 8
      [ok][2021-05-18 13:32:27]
      admin@YF01>
   3. Display alarms and make sure there is no alarm on mgt2 (un-configured management port)
      admin@YF01> show table alarms currentStatus
      ALARM CLEAR
      ID TYPE TIMESTAMP INITIAL TIMESTAMP COUNT DESC
      -------------------------------------------------------------------------------------------------------------------------
      1237 AUTOMATIC 2021-05-18T17:27:53-00:00 2021-05-18T17:27:53-00:00 1 System Policer Alarm Level: Minor, Policer
      [ok][2021-05-18 13:32:30]
      admin@YF01>

Keep track of the management port state changes even if there is no Management IP interfaces on the management ports.
Do not generate management port down event if there is no Management IP interfaces on the management port.

Workaround: None. Optionally, connect unused management ports to Ethernet switch to reflect the port UP status.

The SBC frequently performs a switchover with a core dump after a 9.2.1R0 upgrade.

Impact: An SCM experienced a core dump in the code that is executed only when the STI feature is enabled.

The core dump was the result of code in SipSgStiCopyDisplayNametoCPC() attempting to de-reference a NULL pointer.
The fix is to add a check for NULL before attempting de-reference the pointer.

Root Cause: The root cause is that there is code in an STI specific function that attempted to de-reference a NULL pointer. A NULL pointer check is missing in this code.

Steps to Replicate: Specific steps to reproduce this issue are not known. The root cause and the fix were found through code inspection and core analysis.

The code is modified to add a check for NULL before attempting de-reference the pointer.

Workaround: The only known workaround is to disable STI.

Error in EMA when modifying and saving sipAdapterProfile.

Impact: Error in EMA when modifying and saving sipAdapterProfile.

Root Cause: The scenario comes up when profile name has ellipses in datatable. If profile name is very big, ellipses (...) will be added at the end, due to this the actual profile name is not passed back during save operation.

Steps to Replicate: 

1. Login in to EMA.
2. Navigate to SIP Adaptor Profile.
3. Select the profile that has ellipses (..) and modify then save it.
4. Profile gets saved with any error.

The code is modified to match if ellipses present in profile name, getting the title of selected entry from datatable. The title is not available in profile name without ellipses thus taking text from selected entry.

Workaround: No workaround.

There was a SBC 5400 core dump after MSRP call with direct media.

Impact: A PRS core was encountered when customer was running MSRP with direct media and MSRP Multiplexing was enabled.

Root Cause: The root cause is that the MRM code is using an invalid index to get a pointer to an array element.

Steps to Replicate: This is not reproducible as it is most likely triggered by a race condition.

The code is modified the ensure that MRM uses the correct index when attempting to get a pointer to an array element.

Workaround: The only known workaround is to avoid running MSRP with direct media and MSRP Multiplexing.

The LSWU from V07.02.00-R002 to V09.02.00-R001 has failed with the error "CpxIntfErrorExit".

Impact: Upgrade fails and the SBC fails to come up due to missing SNMP configuration.

Root Cause: The targetAddrsParams for standard V2 trap had been removed from the configuration and this information was expected to be present during the upgrade processing.

Steps to Replicate: Perform upgrade with SNMP configuration in place and check for errors in the CE_Node.log file.

The code is modified to identify the SNMP upgrade failures.

Workaround: Manually add back in targetAddrsParams for traps.

The SBC upgrade failed as the SNMP trapTarget with a name containing white-space.

Impact: If any SNMP trapTarget name contains a space, upgrade to 9.2.1 will fail.

Root Cause: A trap target is configured with a name containing a space character, which is allowed for this table.

Steps to Replicate: On 8.2 software, use CLI to create a trapTarget with space character:
set oam snmp trapTarget "test target" ipAddress 1.2.3.4 state enabled
Perform an upgrade to 9.2.1.

The code is modified so that it can cope with trapTarget names with space characters.

Workaround: Prior to upgrade ensure no trapTarget names have space, recreate with a new name not containing space.

The SBC unable to load config after upgrading from 7.2.3S400 to 10.0A006

Impact: After upgrading a cloud SBC from a pre 9.2.0 version, unable to reload the config when the 'SystemName' is not set to 'vsbcSystem'

Root Cause: The configuration filename pre 9.2.0 uses 'vsbcSystem' instead the of the defined 'System Name', meaning the SBC is unable to find the file to load it.

Steps to Replicate: 

1. Save a configuration on a cloud SBC pre 9.2.0 (e.g. AWS).
2. Upgrade the SBC.
3. Attempt to load the configuration file.

The code is modified to address the issue. 

Workaround: Change configuration filename to be that of the set 'SystemName'.

e.g. If 'SystemName' is set as 'aws-sbc' change:

config_vsbcSystem_20210414_035150.tar.gz -> config_aws-sbc_20210414_035150.tar.gz

PortFix SBX-110639: When the call is dipped the invite sends only the LRN.

Impact: When ingress INVITE's DIVERSION header does not present and TO header is different from RURI, i.e., it has redirecting original number, the egress RURI will take translated number, i.e., LRN, as RURI in egress INVITE, after LNP dip.

Root Cause: The RURI should take LRN in egress, while DIVERSION header in ingress presents. But the code change for a previous issue has made RURI equal to LRN even DIVERSION header was missing.

Steps to Replicate: 

1. Set up SIP to SIP LNP call, on SBC and external PSX system.
2. The TO header username (phone number) is different from RURI.
3. There is no DIVERSION header.

This should reproduce the problem and test the fix.

Please go through all regression test related to SBX-106067. It's important NOT to break any functionalities in SBX-106067.

The code is modified the condition to only set RURI equal to LRN while DIVERSION header presents.

Workaround: None.

Portfix SBX-104733: SCM Process core dump during healthcheck.

Impact: The SCM Process core dumped when too many set operations executed in a single commit.

Root Cause: The Call Processing tasks takes longer than 10 seconds when subscribing to the changes made to SIP Trunk Group.

Steps to Replicate: Configure 12 trunk groups. Modify 11 trunk groups with a single commit command and the CLI will deliver the following error message:
Aborted: 'addressContext default zone ZONE_IAD sipTrunkGroup': Too many set operations between commits. Revert the session and retry(max set operations 10).

Again modify 10 TG in single commit: 
O/P: commit complete

The code is modified per commit is changed to 10 from earlier value of 50.

Workaround: Ribbon recommends performing a commit after every CLI transaction.

PortFix SBX-104619: The FM process coredumped. 

Impact: The FM Process crashed and wrote coredump.

Root Cause: The FM Process tried to read the /proc/meminfo file, which should always exist, but it got a file not found error.

Steps to Replicate: It is not known how to reproduce this as this should never happen, so defensive code added to prevent null read/write.

The code is modified so when we cannot read the /proc/meminfo file, we return the last good value read instead of a NULL to prevent the crash.

Workaround: None.

Portfix SBX-109177: The SbcSftp throws an error as "Failed to add ACL".

Impact: The sbcsftp application does not remove the ACL created correctly.

Root Cause: The permissions to delete the ACL gets lost while lowering permissions to ensure the sbcsftp can only access the current user's files.

Steps to Replicate: 

1. Run SbcSftp as linuxadmin. Verify no 'Failed setting permissions' error messages appear.
2. Verify Cleanup Script:
   1. Run SbcSftp but kill the the process before it completes, with 'kill -9 $PID'.
   2. As root, get the name of the ACL: '$CPSI -d acl | grep SbcSftp'.
   3. Wait 15 minutes.
   4. Verify ACL removed: '$CPSI -d acl | grep SbcSftp'.

The code is modified so that we can raise the permissions again once SFTP is complete.

Workaround: None.

PortFix SBX-109968: The 822R0 SCM Process core dumped.

Impact: The SCMProcess core dumped on a SWe.

Root Cause: Active SCM sent a Redundancy message to Standby SCM with a Registration index that was outside of its MAX range. The error handling code caught this and attempted to clean up the Registration Control Block but had a problem because the Control Block had not yet been initialized.

Steps to Replicate: This problem is not reproducible.

It is most likely triggered by a race condition which results in the Active and Standby to have different numbers for max number of Registrations (on a SWe the max number of Registrations comes from the file callEstimates.txt and in this case, the files on the active and standby do not match).

This appears to be a SWe specific issue (the callEstimates.txt file is only used in SWE setups).

The code is modified to initialize the Registration Control Block immediately after allocating it, before any validation checks, so that the error handling code can clean up the RCB without causing a core.

Workaround: There is no known workaround.

Portfix SBX-110215: A coverity issue.

Impact: Code fix for an SCM coredump (SBC-108237) when the Invite gets a transaction timeout and the 200 Ok for Invite is received at the same time needed some cleanup. Coverity issue: CID:338630

Root Cause: In the problem scenario, the SBC is trying to send ACK for the 200 OK, when the SCM cores while creating a SIP Transaction for ACK Message.

Steps to Replicate: 

1. Run a Basic Invite call flow.
2. Do not answer for Invite on the UAS side.
3. After 32 seconds, send 200 OK for Invite from the UAC side.

A race condition might occur and result in coredump.

The code is modified not to send ACK in this scenario as the CseqType and CseNumber are unknown and 0 respectively.

Workaround: None.

PortFix SBX-101432: Question about the DSP insertion.

Impact: Customer was seeing the "DSP insertion/rejection reason" CDR fields set to "Rejected codec unlicensed" and wanted to know why.

Root Cause: The documentation did not provide any guidance as to when this value could appear.

Steps to Replicate: The customer call scenario is unknown.

The code is modified to provide guidance on when it could happen as per the scenario below: 

These are the cases when DSP Rejection reason can be set to "Rejected codec unlicensed"

1. The SBC does not have a DSP.
2. The SBC has DSP but codec licenses absent.
3. The SBC has DSP and codec licenses but transcoding not enabled for the codecs.

In any of the above cases if passthru is possible and if the call is successful then DSP Rejection reason updates after a successful offer answer.

This value might appear in the case where the offer/answer exchange did not complete.

As the customer was unable to provide details on the specific call flow additional info level logging and call trace logs is modified as well to provide more details for the future.

Workaround: None.

Portfix SBX-108143: Set FAC as enabled by default.

Impact: The FAC was enabled by default in 9.2 but was disabled in release 9.2.1. It is now enabled by default in further releases of 9.2.x and 10.0.

Root Cause: The FAC was temporarily disabled by default until performance issues were fixed.

Steps to Replicate: Run the FAC impacts performance on high cps, and perform load test.

The code is modified so the shared memory is used instead of memory mapped files.

Workaround: Keep the FAC disabled if using high cps until it is upgraded to fixed version.

PortFix SBX-101239: The congestion seen in a SBC with no traffic.

Impact: The SBC 5400 reporting congestion even when no calls active.

Root Cause: On the 5400 server, the EMA has allocated 4 CPUs for java and there are times when these can all run hot even when no calls on the system. The minimum number of hot CPUs to trigger congestion on the 5400 was set to 4 so it could report without any calls.

Steps to Replicate: Enable SIP ladder diagram and CDR viewer on check for congestion reports on an idle system.

The code is modified to avoid congestion when no calls. This is similar to the other 5xxx models where the hot CPUs is one more than the number of java CPUs in EMA.

Workaround: Disable SIP ladder diagram and CDR viewer services to reduce the java CPU usage.

Portfix SBX-105436: The CDR issues for REGISTER.

Impact: There were issues while writing CDRs for REGISTER method.

Root Cause: The issues below were present w.r.t REGISTER CDRs:

1. CDRs were not generated for REGISTER in case of crankback scenarios.
2. TG name was not updated properly for REGISTER CDRs.
3. Disconnect reason was not updated properly for REGISTER CDRs.

Steps to Replicate: 

1. REGISTER an endpoint through SBC with/without crankback.
2. Check CDRs. Entries should be proper.

The code is modified for:

1. Generating EVENT records for REGISTER in case of crankback scenarios.
2. Correcting the egress TG name in the EVENT CDR for REGISTER.
3. Correcting the disconnect reason in the EVENT CDR for REGISTER.

Workaround: None.

Portfix SBX-108198: Coverity issues in nrsPktLifCsv.c

Impact: There was a coverity fix that caused the issue.

Root Cause: Fix added for the Coverity error.

Steps to Replicate: The issue cannot be reproduced.

The code is modified to address the issue.

Workaround: None.

Portfix SBX-108058: The SBC CpxAppProc memory leak.

Impact: During the SBC startup processing there is a small memory leak.

Root Cause: During the startup processing, the SBC is allocating memory while reading configuration information and it is not being freed up correctly at the end of the provisioning steps.

Steps to Replicate: This issue was only observed while running with ASAN images in the engineering lab as the amount of memory leaked is small and cannot be checked.

The code is modified to correctly free the memory allocated while processing the configuration data.

Workaround: None.

Portfix SBX-105856: The Wrong Version in URL after a Restore to an older version.

Impact: The wrong version in URL after a Restore to an older version.

Root Cause: URL is picked from CDB from path "/system/objectStoreParameters/url" that is independent of software version of revision.

Steps to Replicate: 

1. Spawn OAM act, standby and one SSBC with V10.00.00A008.
2. Create multiple revisions (e.g., rev1, 2, 3, 4, 5)
3. Upgrade cluster to V10.00.00A009.
   
   [root@VSBCSYSTEM-2-vsbc2 172.16.10.52 evlog]# cat /mnt/gfsvol1/config-versions.txt
   5,V10.00.00A008,oam_config_20210510T042937.tar.gz,0,2021-05-10T04:29:37,,downloaded_from_ems
   6,V10.00.00A009,oam_config_20210510T043100.tar.gz,1,2021-05-10T04:31:00,REBOOT_REQ,auto_save_after_software_change
   [root@VSBCSYSTEM-2-vsbc2 172.16.10.52 evlog]#
4. Manually reboot and replace the SM Process with a fix.
5. Restore revision 3.
   
   7,V10.00.00A008,oam_config_20210510T045034.tar.gz,6,2021-05-10T04:50:34,RESTORE,restored_from_3
   8,V10.00.00A009,oam_config_20210510T045311.tar.gz,6,2021-05-10T04:53:11,REBOOT_REQ,auto_save_after_software_change
   
   Observation: Revision 7 uploaded with correct URL.
6. Remove revision 7 from /mnt/gfsvol1 and restore revision 7.
   Revision successful and newly created revision uploaded on EMS with correct URL
   11,V10.00.00A008,oam_config_20210510T055545.tar.gz,20,2021-05-10T05:55:45,RESTORE,restored_from_9
   12,V10.00.00A009,oam_config_20210510T055745.tar.gz,20,2021-05-10T05:57:45,REBOOT_REQ,auto_save_after_software_change

The code is modified to update the URL with software version of revision being restored.

Workaround: None.

PortFix SBX-108356: The SBC has various runtime errors found in np.log.

Impact: Internal ASAN builds report runtime errors on SWe platform related to left shift of signed integers.

Root Cause: Appropriate integer casts were missing in code, which caused ASAN runtime warnings related to bit shift.

Steps to Replicate: Bring up ASAN build on SWe platform and observe if np.log contains ASAN runtime warnings related to bit shifts.

The code is modified to fix the runtime warnings in ASAN builds.

Workaround: No workaround.

Portfix SBX-102925: The issues in Ova\QCow2 installation

Impact: The SBC SWe installed through the ISO and SBC SWe created through the image launch method have different prompts and root password.

Root Cause: While installing through the image launch method, the SBC SWe was getting configured the cloud SBC way w.r.t root password and prompt.

Steps to Replicate: 

1. Install the SBC SWe using the ISO. Check the root password and CLI prompt.
2. Instantiate the SBC SWe using Image launch. Check root password and CLI prompt.

Results from step 1 and step 2 should be same.

The code is modified to ensure both image launch method and ISO install method of the SBC SWe has no root password and same CLI prompt.

Workaround: The user can manually remove root password on ISO installed SWEs using below command:
usermod root -p '*'

Portfix SBX-110179: The LeakSanitizer: detected memory leaks.

Impact: A memory leak occurs when a logical management interface is added or modified.

Root Cause: A confd cursor element was not closed when exiting the routine that validates the logical management interface being added or changed and this resulted in a memory leak.

Steps to Replicate: Make changes to the logical management interfaces and check for memory increasing in the Cpx process.

The code is modified to ensure the associated memory is freed to avoid the leak.

Workaround: None.

PortFix SBX-105763: Move the dmesg monitoring from SM to a platform cron job

Impact: Move dmesg monitoring from SM to a platform cronjob

Root Cause: Since the dmseg can be large for long-running SBCs (thus, take longer to dump logs), the function can take longer causing an SM healthcheck.

Steps to Replicate: Check for "/var/log/sonus/tmp/dmesgErrorMarker.tmp" after manually running the script

The code is modified to run every minute as a cron job to find i/o and filesystem errors in dmesg. If error is found, it'll create a marker file in tmp, which can be later used by other script to check sanity of the system.

Workaround: None

Portfix SBX-94852: Security, Audit and other logs modifiable (including deletion).

Impact: Administrator users are able to delete or modify evlog files on the SBC.

Root Cause: Users belonging to upload group(like admin) had write access on the evlog dir, which allows them to delete/modify log files.

Steps to Replicate: Use SFTP to login to the evlog directory as user admin, and attempt to modify/delete log files.

The code is modified that prevents the admin from having writing access on the files owned by other users.

Workaround: None.

Portfix SBX-108435: The SBC CpxApp Process dumps core in the standby OAM. 

Impact: The CPX coredump occasionally writes when the SBC is stopped.

Root Cause: Replication Engine thread is not stopped when the CPX receives deactivate request.

Steps to Replicate: It is rarely reproducible.

The code is modified to address the issue.

Workaround: None.

Portfix SBX-106543: The SBC experiences a core dump while running UAS Notify Request.

Impact: The SBC coredumps while processing the UAS Notify Request within the XML body.

Root Cause: In API SipsCheckForAnyBody(), the loop where the string pointer pch is incremented each time was being accessed before validating for an out of bounds condition that caused the crash.

Steps to Replicate: Send a NOTIFY with XML body from UAC towards the SBC.

The code is modified to add a out of bounds check before accessing the pointer value.

Workaround: None.

Portfix SBX-108370: The top2 on the SBC core takes lot of CPU cycles.

Impact: The SBC performance monitoring tools like top2 at times take 20% of a CPU core there by reducing total available CPU resources for management activities on SBC.

Root Cause: Introduced nice values to sbxPerf commands but still top2 taking around 20-30% at times. As a result, need to check ways to optimize it to take less CPU using /root/.top2rc.

Steps to Replicate: Install fix build and by using top command make sure CPU utilization of top2 should not be much CPU utilized.

The code is modified to take less CPU utilization of the SBC performance monitoring tools like top2.

Workaround: None.

Portfix SBX-109681: Low MOS score for UNENCRYPTED_SRTP calls

Impact: In SRTP for unencrypted, Authenticated case, SRTP packets were being discard at NP due to authentication key mismatch.

Root Cause: In SRTP for unencrypted, authenticated combinations key size is required in NP to derive the session authentication keys. Since the cipher key size was not being pass to NP, session authentication key was wrongly calculated.

Steps to Replicate: On the receipt of an SDP offer with crypto attributes, if 'enable SRTP' is Configured in packet service profile, a common crypto suite is found in the crypto attribute received in the offer and in the packet service profile, and session parameters if included are allowed, the offer will be accepted. In the answer, the SBC includes the same tag used in the offer.

Test Setup on the SBC:

1. Endpoint1 Configured with SRTP profile( SRTP/SHA-1-80). Disable all Session Parameters in packet service profile. Allow Fallback enabled.
2. Endpoint2 Configured with SRTP profile(SRTP/SHA-1-80).Enable Session Parameters UNENCRYPTED_SRTP in packet service profile. Allow Fallback enabled.

Procedure:

Endpoint1 sends an offer SDP with crypto attribute SRTP/SHA-1-80 and with Session Parameters UNENCRYPTED_SRTP
Endpoint2 replies with crypto attribute SRTP/SHA-1-80 and with Session Parameters UNENCRYPTED_SRTP

The code is modified so the SBC application is passing cipher key size also to NP to calculate session authentication keys.

Workaround: Workaround is to not send unencrypted SRTP. Use cases with Encrypted SRTP and authentication no issue is seen.

Portfix SBX-105900: Resize log volume on every boot.

Impact: The log volume is not being resized on every boot.

Root Cause: Currently, we build the filesystem on cinder volume only on the first boot.

Steps to Replicate: 

1. Create and attached a cinder volume of 50 GB.
2. Bring down the SBC, resized cinder volume to 100 GB.
3. Launch the SBC with the cinder volume attached.

Check if the cinder volume is resized to 100 GB.

Check if file system is already built. If the file system is not built, resize the volume to address the issue.

Workaround: None.

Portfix SBX-108190: SBC: The callTracing does not work after reverting a switchover.

Impact: You cannot enable the callTrace on calls after a switchover under the following circumstances: When maximum calltrace count is reached before the switchover, and all calls with calltrace enabled are terminated after the switchover.

Root Cause: The internal calltrace state was not properly synchronized to the standby node that caused no new calls with calltrace ON request can be enabled.

Steps to Replicate: 

1. On a 1+1 (active/standby) SBC set up, set the calltrace count to 10.
2. Create 10 calls with calltrace enabled.
3. Perform a switchover.
4. Terminate the 10 calls after switchover.
5. Perform a switchover.
6. Make new calls with calltrace ON requested in the ADD termination command.

These new calls should have calltrace enabled.

The code is modified so that calltrace works after a a switchover.

Workaround: Reboot both SBC nodes.

PortFix SBX-109084: The IPv6 SBC traps was not received to the EMS.

Impact: For trapTargets, certain IPv6 addresses are sent to the incorrect IPv6 address.

Root Cause: If the IPv6 address, converted to the form of 16 decimal octets separated by periods have a length greater than 47 digits, the address will be truncated. 

Steps to Replicate: 

Provision an OAM SNMP trapTarget with a customer IP.

Observe the tailf snmp.log that the actual address sent to.

The code is modified so the buffer used to store the converted IPv6 string is stored in a buffer of 64 characters, which accommodates any IPV6 address.

Workaround: None.

Portfix SBX-109862: The SBC is not rejecting the SRTP call when disallowSrtpStream is enabled.

Impact: The SBC is not rejecting the SRTP call when disallowSrtpStream is enabled in ingress PSP and Invite is received with only SRTP stream.

Root Cause: This is specific call flow when disallowSrtpStream is enabled and an INVITE is received with only SRTP stream. This scenario was not handled and the SBC was not rejecting call.

Steps to Replicate:

1. Feature flag "multipleAudioStreamsSupport" under trunk group media is enabled for both ingress and egress.
2. A disallowSrtpStream flag under trunk group should be enabled on the ingress side.
3. Make a call with two SRTP stream.

The code is modified so that when the disallowSrtpStream is enabled and an INVITE is received with only an SRTP stream, the call is rejected with 488.

Workaround: None.

Portfix SBX-109966: The SBC incorrectly accepts an SDP offer in an ACK

Impact: The SBC is designed to ignore an SDP offer in an ACK, but is not doing so.

Root Cause: In an early media call, the SIP stack was accepting the SDP in ACK as an offer. This should never happen.

Steps to Replicate: 

1. In an early media call, add SDP in ACK.
2. Send a re-INVITE with SDP from the same side.
3. SIPS stack should ignore the SDP in an ACK and accept the SDP from re-INVITE as an offer.

The code is modified to not forward the SDP to the application.

Workaround: None.

PortFix SBX-109298: SBC: The DSP fails to modify ptime.

Impact: When the SBC receives a re-INVITE with a change in the ptime for EVS, the DSP Modify command fails.

Root Cause: Handling of DSP Modify Command for ptime change for EVS was not present.

Steps to Replicate: Run a EVS to G711 call, send a re-invite on the EVS leg with change in ptime. The change in ptime should be put into effect through a DSP Modify.

The code is modified to support for ptime change for EVS is added. Allowable ptime changes include 20, 40, 60, 80 and 100ms.

Workaround: None.

Portfix SBX-106127: The SBC product name is incorrect in a 10.0 SBC.

Impact: The sbcDiagnostic incorrectly prints product name as "ConnexIP5000" instead of "AWS".

Root Cause: Platform type is determined by querying platform data using dmidecode command, due a bug in the query platform type is returned as unknown. As a result of this bug, the sbcDaignostic shows generic product name ("ConnexIP5000").

Steps to Replicate: Run sbcDiagnostic command from the Linux shell of the SBC. It shows the following:
************SBC Information *********
.......
.....
SBC Product Name: ConnexIP5000

After a fix, it prints the following: 
************SBC Information *********
.......
.....
SBC Product Name: AWS

The code is modified to return correct platform type.

Workaround: No workaround.

PortFix SBX-109993: The PRS Process is coring with a pkt switchover with a loopback call.

Impact: The PRS Process core dumped when testing a pkt port switchover.

Root Cause: The statement to log the debug message was missing a string parameter.

Steps to Replicate: Force pkt port switchovers. This is intermittent so may be difficult to reproduce.

The code is modified to address the issue.

Workaround: No workaround.

Portfix SBX-108956: SBC: Default bitrate for G722 codec should be 64kpbs in SBC but it is 48 kbps.

Impact: The SBC is using 48kbps bit rate for G722 codec instead of 64kpbs when setting up G722 calls.

Root Cause: The 48kbps bitrate is incorrectly chosen for G722 codec, resulted in the media packets being generated with 48kbps bitrate.

Steps to Replicate: Make one g722 to g711 transcode call and observing the RTP stream for g722 codec.

Changed bitrate to 64kpbs when setting up G722 calls to address the issue.

Workaround: None.

Portfix SBX-107975: A Serf event processor is unable to restart because restart check marker file is not getting removed.

Impact: The Serf event processor is unable to restart because the restart check marker file is not getting removed.

Root Cause: The restart check marker was only being removed if serfeventProcessor starts successfully, so if it fails to start, any attempts to restart it would be prevented because the marker is not removed.

Steps to Replicate: 

1. Make serf event processor fail at some point.
2. It should try to restart up to 5 times if it keeps failing to come up.

The code is modified so that the user can attempt a restart.

Workaround: None.

Portfix SBX-105391: SBC: The SM process leaks memory on an OAM active for an SBC switchover.

Impact: While carrying out operations like a configuration upload to EMS, the memory is leaked.

Root Cause: The Python/C APIs cause a memory leak while using functions to the upload/download configuration from the EMS.

Steps to Replicate: Create a direct single instance(ASAN build) registered with EMS, carry out operations saveAndActivate/restoreRevision and change glog/sanitizer_SmProcess* and verify no leaks due to above operation.

Change the implementation from using Python/C APIs to libcurl.

Workaround: None.

Portfix SBX-109167: The AddressSanitizer: SEGV on unknown address 0x000000000028.

Impact: During the pre-parsing the Messagebody, the SEGV on an unknown address is observed in SipsPreParseMsgBody(). This could result in an SCM coredump.

Root Cause: When the Content-Type is NULL the code was performing string comparisons to see if its an expected Content-Type and the code did not verify that the header pointer was not NULL before accessing it.

Steps to Replicate: 

1. Make an A to B call.
2. Send a re-INVITE with the MessageBody Content-Type as empty.

The code is modified to check the Content-Type string pointer is not NULL before accessing it.

Workaround: None.

Portfix SBX-105644: The sonusSbxCertificateExpireSoonWarningNotification trap does not display for all certificates in the current and history alarm lists.

Impact: The sonusSbxCertificateExpireSoonWarningNotification trap does not display the individual alarms for separate certificates.

Root Cause: The SBC alarm for sonusSbxCertificateExpireSoonWarningNotification was just using trap ID as key identifier.

Steps to Replicate: Add multiple certificates to a 1:1 HA system that will expire and configure certificate expiry date. Confirm:

• Only one trap is triggered per certificate.
• One alarm exists for certificate.

The code is modified to use certificate name as well as trap ID as key identifiers to show the alarm.

Workaround: None.

PortFix SBX-109407: MicroFlows are failing in the REGISTER Performance of 2400 RPS (256000 REGISTRATIONS) with an error code 0xf9.

Impact: Unable to support the 256k concurrent subscriber registrations in Default memory profile for the SBC SWe.

Root Cause: Existing logic for determination of flow hash causes increased number of hash collisions leading to increased depth of hash buckets, leading buffer starvation.

Steps to Replicate: 

1. Spawn a SBC SWe instance (of default memory config profile).
2. Test the 256K registrations.

The code is modified to minimize hash collisions.

Workaround: There is no workaround for this.

Portfix SBX-109424: The SBC sends a 420 Bad Extension response when an INVITE with both supported and require 100rel is sent.

Impact: The SBC sends 420 Bad extension when Require:100Rel is received in initial Invite and e2e Prack flag is enabled

Root Cause: Incorrect code was added to reject an INVITE with Require:100Rel when rel100Support flag is disabled and e2e prack flag is enabled.

Steps to Replicate: 

1. The rel100Support must be disabled.
2. The e2e Prack is enabled.
3. An INVITE is received with a Require:100Rel Header.

The code is modified for rejection in the require:100rel scenarios.

Workaround: Use SMM to remove Require:100Rel Header.

Portfix SBX-105437: The CDR issues for non-INVITE messages when blacklisting is involved.

Impact: In case of handling of failure responses for non-INVITE messages, before writing the CDR for a current failure cause code, the SBC was finding out next route and sending a message on network.

This worked fine in normal cases as after sending out a request, the response was processed later, after writing a CDR for current failure response.

However in a backlisted entry case, no actual message is sent out, so the blacklisted entry CDR was written before the previous CDR response code.

Root Cause: Whenever a blacklisted entry was involved, the CDR entries were inaccurate for this blacklisted entry and the previous entry.

Steps to Replicate: Configure Routes for non-INVITE as follows:
R1
R2 - > Blacklisted
R3
R4 -> Blacklisted

The CDR's should be printed in order R1, R2, R3, R4 after a fix.

The code is modified to write the CDR for the current failure response code, and later find next route and send a request on the new route.

Workaround: None.

PortFix SBX-105175: The SBC sends a re-INVITE while media is played on the ingress side in a GW-GW early media call.

Impact: In a GW-GW call scenario, while the media is played on the ingress Gateway, the egress SBC is sending a re-INVITE to the UAS.

Root Cause: Before the ingress GW completes its end to end activation, it received a MODIFY OFFER request from the egress GW due to the change in SDP received in 200 OK for lockdown INVITE. This caused the ingress GW to process modify offer first and then end to end activation later that triggered a re-INVITE towards UAS.

Steps to Replicate: 

1. Send INV from UAC to GW1.
2. GW2 sends INV to UAS.
3. UAS sends 180 SDP.
4. GW1 sends 180 SDP to UAC and starts to play tone.
5. UAS sends 200 OK with out SDP.
6. GW1 sends 200 OK to UAC.
7. GW2 sends ACK and triggers a lock down INVITE.
8. UAS sends 200 OK with change in SDP.

The code is modified so that while modify offer-answer is handled properly during end to end activation.

Workaround: None.

Portfix SBX-109418: The LeakSanitizer detected memory leaks Direct leak of 883820 byte(s) in 413 object(s).

Impact: The SBC was not freeing memory in one of the failure cases.

Root Cause: The SBC was not freeing memory in few cases where incoming INVITE handling fails.

Steps to Replicate: 

1. Send an INVITE with Replaces having call id, to in the Replaces header that does not match to any existing leg.
2. After the call ends, the ASAN should not show memory leak.

The code is modified to free up memory allocated, in all cases when the INVITE handling fails.

Workaround: None.

Portfix SBX-107960: The AWS IPs remain assigned to the standby SBC. Unnecessary dual restarts.

Impact: If communication between active and standby is broken (over ha0 interface) both assume active roles (split brain). When this happens, the standby node that becomes active calls AWS APIs to move IP address to self. Once ha0 link is restored, the machine that becomes active does not call AWS API to move IP addresses, and this might cause issue when node that has IP does not come up as active. In this case, IPs are assigned on standby and another node becomes active.

Root Cause: Root cause of this issue is not calling AWS switchover API (move IPs) during split brain recovery.

Steps to Replicate: Perform a Split brain test and recovery of the SBC HA, and verify that API query is send by an Active SBC after recovery.

The code is modified to call AWS APIs to move IP to current active machine during split brain recovery path.

Workaround: Manually move all secondary IPs to current active machine to restore calls.

PortFix SBX-108434: SBC: A Standby OAM fails to come up after the restart of active and standby.

Impact: After restarting the active and standby or after a fault that causes the SBCs to go down, the standby OAM waits for active to come up first and never recovers if active is down.

Root Cause: On the Standby OAM startup sequence, there is a makeSureActiveIsUp() loop that exits only after active is up. This results in standby to be in a hung state forever if active is down.

Steps to Replicate: 

1. Start active OAM SBC.
2. Start the standby OAM SBC after 1 minute.
3. Stop the active OAM SBC before it becomes active.
4. The Standby OAM SBC should reboot and recover after 10 minutes.

The code is modified to reboot the instance that is starting as standby, if peer does not become active in 10 minutes.

Workaround: Reboot the standby OAM SBC to fix the issue.

PortFix SBX-109017: SBC: Observed MAJOR logs for BRM "BrmAsyncCmdErrHdlr" on T140 load.

Impact: Occasionally enable-RID errors are seen in NP when the system is subjected to large number of call flows that employ the RTCP termination.

Root Cause: The message drops in internal queues due to momentary congestion.

Steps to Replicate: Run an SBC AMRWB<>T140 full load with the RTCP enabled.

The code is modified to ensure the control messages experience a larger queue depth while traversing internal queues.

Workaround: No workaround.

Portfix SBX-107646: For a revision not present in the OAM or EMS, upon doing a restore, an error should be thrown.

Impact: Not showing the proper failed message when failing to download from the EMS.

Root Cause: The confd was not waiting for the EMS download error as the EMS download was done in a different thread context.

Steps to Replicate: Case 1:

1. Create 3 revisions on the EMS.
2. Delete the second revision on the OAM and EMS.
3. Request the system admin vsbcSystem restoreRevision revision 2.
   o/p
   admin@Rahul-OAM1-192.168.20.13% request system admin vsbcSystem restoreRevision revision 2.
   This command will restart all nodes unless the target revision is for a previous version of software. Do you want to continue [yes,no] yes
   result failure
   reason bundle not found locally or on EMS, unable to view changes for this revision.
4. See the above failure message and no restart of nodes.

Case 2:

1. Create 3 revisions on the EMS.
2. Delete the second revision on the OAM.
3. Request the system admin vsbcSystem restoreRevision revision 2.
4. Will observe a restart in all nodes.

Case 3:

1. Create 3 revisions on the EMS.
2. Delete the second revision on the EMS.
3. Request the system admin vsbcSystem restoreRevision revision 2.
4. Will observe a restart in all nodes.

Any other failure case.

The code is modified to run the restoreRevision procedure on the same thread context. This helps to display proper error in case of failure.

Workaround: None.

Portfix SBX-110178: The PRS Process gave heap use after free on address on latest 10.0 build.

Impact: The PRS Process gave a "heap use after free on address" error while running the HA suite on ASAN build.

Root Cause: Interface number was being returned after typecasting the main structure to packet LIF structure, and not management LIF structure.

Steps to Replicate: Run SBX_504_HA suite on ASAN build.

The code is modified to correct the typecasting.

Workaround: None.

Portfix SBX-110128: AWS CFN: The SBC auto-registration in the EMS is not working with EMS FQDN.

Impact: The SBC auto-registration in the EMS is not working when using EMS FQDN.

Root Cause: A service discovery is unable to resolve the EMS FQDN using a system DNS because there is no ACL rule to allow DNS query to go through.

Steps to Replicate: 

1. Configure the SBC with EMS FQDN.
2. Ensure that it is registering to EMS successfully.

The code is modified to add an ACL rule to allow DNS query to go to the configured nameserver.

Workaround: None.

Portfix SBX-104319: There is a conflicted design between two features “sdpRelayAttribute” and “Send RTCP Bandwidth Info”.

Impact: The SBC sends duplicate b=RR and b=RS attributes when “sdpRelayAttribute” and “Send RTCP Bandwidth Info” are enabled together.

Root Cause: The SBC skips adding RR/RS to SDP due to "sendRTCPBandwidthInfo" only when “sdpRelayAttribute” is enabled with TFT.

Due to this, b=RR and b=RS attributes are added twice in outgoing SDP, when “sdpRelayAttribute” was enabled without TFT.

Steps to Replicate: Configuration:

1. Enabled sdpAttributesSelectiveRelay flag under TG.
   set addressContext default zone ZONE_V6 sipTrunkGroup TRUNK_V6 media sdpAttributesSelectiveRelay enabled
   set addressContext default zone ZONE_V4 sipTrunkGroup TRUNK_V4 media sdpAttributesSelectiveRelay enabled
2. Enabled RTCP with below settings.
   set profiles media packetServiceProfile DEFAULT rtcpOptions rtcp enable rrBandwidth 500 rsBandwidth 150

Procedure:

1. Make a Basic call such the the bandwidth parameters are received from the endpoint as b=RR:1125, b=RS:775

Expected Result:

1. The SBC should transparently send the b=RR & b=RS parameters in SDP and should not honor the bandwidth configured in the PSP.

Also, the SBC should not add duplicate b=RR & b=RS parameters in SDP honoring the configured bandwidth values in PSP.

If “sdpRelayAttribute” is enabled without TFT, and "sendRTCPBandwidthInfo" is also enabled, the SBC does not add the RR/RS due to "sendRTCPBandwidthInfo" since attributes are relayed.

Workaround: None.

Portfix SBX-110201: A late offer call resulting in the SBC not sending DTMF for AMR-WB in initial offer towards ingress.

Impact: The SBC is not sending 16k 2833 Payload type in the initial offer towards ingress during a Late media "convert" call.

Root Cause: Answer received from egress contained both 8k and 16k 2833 Payload type and that resulted in the SBC incorrectly assigning the 8k PT value to 16k as well while generating offer towards ingress. As a result, the 16k PT get dropped by SIP stack.

Steps to Replicate: Configuration:

Transcode conditional
rel100Support enabled
honorSdpClockRate enabled
DLRBT enabled
Configure multiple codecs on both Routes

To re-create the issue:

1. The UE initiates Late media convert call.
2. The SBC sends out 180 answer with the below SDP:
   
   m=audio 1024 RTP/AVP 96 8 97 0 18 9 101
   a=rtpmap:96 AMR-WB/16000
   a=fmtp:96 mode-set=0,1,2; mode-change-capability=2; max-red=0
   a=rtpmap:8 PCMA/8000
   a=rtpmap:97 AMR-WB/16000
   a=fmtp:97 mode-change-capability=2; max-red=0
   a=rtpmap:0 PCMU/8000
   a=rtpmap:18 G729/8000
   a=fmtp:18 annexb=no
   a=rtpmap:9 G722/8000
   a=rtpmap:101 telephone-event/8000
   a=fmtp:101 0-15

Test Result without a fix:

The SBC drops the telephone-event/16000 payload type when generating offer towards ingress in 180.

The code is modified to prevent the same PT value getting assigned to both 8k and 16k DTMF in this call flow.

Workaround: None.

Portfix SBX-106693: SBC: There was a wrong warning message on the Direct-SBC while doing restore.

Impact: The wrong warning message was displayed during a restoreRevision.

Root Cause: Rewording of the text message required for when restorerevision was invoked.

Steps to Replicate: 

1. A new image was created after changes.
2. The new csar file was loaded to VNF Manager.
3. Created revision 2 on the EMS.
4. Run the command "request system admin vsbcSystem restoreRevision revision 1".
   test output
   admin@Rah-OAM1-192.168.20.3% request system admin vsbcSystem restoreRevision revision 1
   This command will restart all nodes unless the target revision is for a previous version of software. Do you want to continue [yes,no]

The code is modified by rewording the warning text message.

Workaround: None.

PortFix SBX-110362: The SBC generates RTP inactivity alarms when the policer mode is set to bypass.

Impact: The SBC generates false RTP inactivity alarms when the policer mode is set to bypass.

Root Cause: A bug in the media policing logic of SWe_NP code skips updating the timestamp of last RTP packet of a media flow, resulting in raising RTP inactivity notifications for every 10 seconds.

Steps to Replicate: 

1. Configure a ISBC instance.
2. Set the policer mode to bypass (Unit tested by hacking the code).
3. Configure RTP inactivity.
4. Run the call for more than the rtp inactivity duration.
5. Observe that RTP inactivity alarms being raised, even if traffic flow is happening.

The code is modified to update the timestamp after every arrival of RTP packet for the media flow.

Workaround: Ensure the policer mode is set values to other than 'bypass".

Portfix SBX-110054: The encrypted packets not being sent towards racoon in case of IPsec tunnel mode.

Impact: The lawful Interception fails when IPsec is enabled for LI media over UDP.

Root Cause: An earlier fix assumed the LIG information to be stored in the selector structure to send notification to XRM, indicating if a IPsec tunnel is for LI media.

The field was LIG was present in the selector but was never updated.

This resulted in failure to send notification to XRM to indicate IPsec tunnel for LI media, for it always failed the match with the LIG configured under the CDC and as a result, the LI failed.

Steps to Replicate: 

1. Generate Local and remote Certificates which has FQDN as a SAN value and signed by an Intermediate CA.
2. Copy Intermediate CA's and Racoon's .crt files to /opt/sonus/external on SBC.
3. Covert Remote Files to .der and local files to .p12.
4. Configure IPSEC related configuration on SBC.
5. Execute a basic call over IPSEC interface.

Expected Results: 

1. CA, Local(SBC) and Remote(Racoon) certificate generated successfully.
2. Copy of certificate files to SBC  is successful.
3. Conversion of remote and local certificate is successful.
4. All configuration issuccessful.
5. IMS LI is configured successfully.
6. Basic call must run successfully over ipsec enabled interface. Signaling and media are captured on IMS LI

The LIG check is removed to ensure that the notification is sent to XRM to indicate the IPSEC Tunnel is for LI media.

Workaround: No workaround.

PortFix SBX-102618: The SBC needs to handle the scenario where member-join event handling is missed at the serf level.

Impact: A race condition in RGM event handling can show sync to be inProgress in N:1.

Root Cause: A race condition in handling member-join event causes some member-join events to get discarded at serf level.

Steps to Replicate: Perform multiple sbxrestarts but it is not readily reproducible.

The code is modified to ensure the Redundancy Group is up-to-date.

Workaround: None.

Portfix SBX-106589: The SBC SWe does not forward the SIP Info and drops the call.

Impact: After receiving 32763 SIP indialog INFO messages, the SBC sends a 500 error for further INFO messages instead of forwarding the message.

Root Cause: The SBC maintains the callHandle structure with refcount, which will be incremented for every reference and decremented once reference is released. But in this case, the refcount is not decremented properly, for every indialog msg (INFO) request received, the count kept increased.

Steps to Replicate: 

1. Establish a call.
2. Send indialog INFO messages more than 32763.

The code is modified to address the issue.

Workaround: No workaround.

Portfix SBX-106897: The SBC does not send notify for successful trace activation.

Impact: The SBC does not send a calltrace notify to C3 in a call setup with calltrace ON request for ADD commands of both first termination and second termination of the call, but the ADD command failed for the second termination.

Root Cause: The SBC code logic was to send a calltrace NOTIFY after the ADD commands of both termination have been processed, but this result in calltrace NOTIFY not sent for the first termination in failure of ADD in the second termination.

Steps to Replicate: 

1. Enable calltrace in the SBC.
2. Send ADD request with CALLTRACE/TRACEACTIVITYREQUEST=ON on both the terminations. The codec should be PCMU on the first termination and G726 on the second termination, such that ADD of second termination will fail (not supported codec).

Expect Result:

1. The call fails.
2. The SBC sends calltrace NOTIFY with RES=success for the first termination.

The code is modified to send calltrace NOTIFY for successfully ADDed termination.

Workaround: None.

Portfix SBX-109439: SBC: An activeRevision failure when state for even type audit is set to on/off.

Impact: The configuration Playback on managed VMs fails on command.

set oam eventLog filterAdmin vsbc1 audit audit level major state off

Root Cause: The playback engine does not use user context.

Steps to Replicate: Run the command on the OAM.

set oam eventLog filterAdmin vsbc1 audit audit level major state off
commit

Perform a saveAndActivate.

The code is modified to address the issue.

Workaround: Reboot on managed VMs to get the configuration at startup.

Portfix SBX-109412: The destination buffer was too small during the snprintf function.

Impact: During the snprintf function call, the destination buffer was not big enough to copy the source string.

Root Cause: The size of the destination buffer was smaller than the source buffer because the destination buffer length check was not present.

The buffer file size should be 256 bytes. File: call/sonus/p4/ws/jenkinsbuild/sbxAsan100/marlin/SIPS/sipsParse.c

Steps to Replicate: 

1. When the callId length was more then the defined max callId length this error was seen.
2. Run a call with callId length greater than 256 bytes. This error should not come.

The code is modified to address the issue.

Workaround: None.

Portfix SBX-109453: The SBC is closing the socket for DNS query over TCP frequently.

Impact: The SBC is closing the socket for DNS query over TCP frequently.

Root Cause: Whenever the TCP connection is closed, the FIN packet is sent towards the SBC from DNS server. But the application was not closing the connection by sending the FIN and Even after a connection is closed by DNS server, the application is still holding socket information. This details were used in future queries has cause TCP connection to reset.

Steps to Replicate: 

1. Have a v6 Cloud SBC with the below build
   SBC: V10.00.00-A006.
2. Configure the DNS server to send queries using TCP connection.
3. Make a call.
4. Wait for some time and run one more call here.

Actual behavior:

In step 3, the SBC will query DNS server to resolve NAPTR records over TCP connection.

In step 4, the SBC will try to send a NAPTR query and then immediately close TCP connection.

Expected result:

The NAPTR query should be successful in step.

The code is modified to close TCP connection (Sending FIN to close connection). Also, the connection information in application is freed.

Workaround: No workaround.

PortFix SBX-106520: Among the exported setting values, there were setting values that were not set in other environments.

Impact: The "comment" parameter is missing from some configuration related to AAA rules/cmdRules.

Root Cause: The "comment" parameter is not being restored during a LSWU.

Steps to Replicate: 

1. Spawned a SBC, exported the configuration and observed comment parameters are present.
2. Upgraded the SBC software. exported the configuration and observed comment parameters are not present.
   
   Revised the steps above with fix, and the issue was not observed.

The code is modified to restore "comment" parameter during a LSWU.

Workaround: None. The "comment" parameter is used only for description. It doesn't have functional impact.

Portfix SBX-106869: The SBC Node branch information is inconsistent after a Cleanstart/Restore Revision is performed in OAM's.

Impact: The SBC Node branch information is inconsistent after the Cleanstart/Restore revision is performed for OAMs.

Root Cause: Managed VMs registration failed with active and standby OAMs as OAMs were not up. VMs did not retry to register again.

Steps to Replicate: After the issue is fixed, create a setup OAM and S/M/T SBC.

1. Log in to the CLI as admin and execute the following:
   show configuration node
   o/p: All nodes should be listed.
2. Configure the SBC (create AddressContext, zone, TG) saveAndActivate Revision
   o/p: new configRevision should be propagated to all managed VMs.

The code is modified to keep retrying to register with active and standby OAMs.

Workaround: None.

PortFix SBX-108112: MS TEAMS - The Media Stats are not correct when the DLRBT profile is removed from TGs and ICE is enabled.

Impact: On an SBC was configured for MS Teams LMO centralized mode with ICE, if DLRBT is not correctly configured, this can lead to media not flowing correctly for a call even after ice learning gets completed.

Root Cause: The early ICE learning logic for non DLRBT mode in the SBC was not fully deactivating ICE media resources on receipt of 200 OK . As a result, the resources were unable to be re-activated for end to end media flow.

Steps to Replicate: On an SBC configured as MS Teams LMO centralized mode with ICE on MS Teams TG:

1. Disable DLRBT on PSTN and MS Teams TG's.
2. Establish a PSTN endpoint to MS Teams call that uses primary (Internal) LIF address towards Teams.
3. Once call has established and ice learning has completed, send media (voice) in both directions between PSTN and Teams endpoints.
4. Media should flow as expected in both directions and call Media status should correctly show the number of media packets sent and received for ingress and egress.

The code is modified to correctly deactivate early ice learning media resources on receiving an SDP from MS Teams.

Workaround: The DLRBT should be enabled.

PortFix SBX-109220: The monitor.c "runtime error: signed integer overflow: 0 - -9223372036854775808 cannot be represented in type 'long int'" error in the np.log.

Impact: Internal ASAN builds report runtime signed integer overflow error in SWe for standard deviation metric for jitter meant for consumption by Ribbon Protect.

Root Cause: Existing algorithm of standard deviation metric did not handle integer overflow issues.

Steps to Replicate: Stream the custom RTP packet where expected standard deviation and verify via Ribbon Protect metric that computed standard deviation values are in range.

The code is modified to handle the overflow.

Workaround: No workaround. This metric is only exposed to Ribbon Protect. This is not exposed as a metric in any SBC display or CDR.

PortFix SBX-70800: AWS:Observing that metaVariable table is getting modified on the loading the backup configuration file of one instance in other instance.

Impact: When loading the backup configuration from one SBC instance to another, the metavar table is getting populated with the list of metavars from both instances. This by itself does not cause any issues so long as the metavars are unique, it is incorrect to see the metavars for another instance in the table.

Root Cause: The code was not removing the existing metavars prior to loading the configuration of another instance. This meant the metavar table had both sets of information.

Steps to Replicate: 

1. Create a backup configuration of one cloud instance (instance1).
2. Loaded the backup configuration file into a new cloud instance (instance2).
3. Check the metaVariables table and see if it contains the metavars from both instances.

The code is modified to flush the metavars for the existing instance prior to loading the configuration from another instance.

Workaround: None.

PortFix SBX-108577: The CE_2N_Comp_SmProcess==11649==ERROR: AddressSanitizer: SEGV on an unknown address 0x000000000000.

Impact: The SBC was performing a write operation on one of the un-allocated memory space while restoring NTP server configuration. As a result, SEGV on unknown address was reported.

Root Cause: This issue was caused because a flag variable was not initialized. As a result, if condition was evaluated true instead of false, a write operation would be performed.

Steps to Replicate: Configuring the NTP server and restoring the configuration by switchover or restart this error should not come up.

The code is modified to address the issue.

Workaround: None.

Portfix SBX-106759: In a N:1 mode, the SBC performs a switchover for a different node than the switchover request is honored for. 

Impact: In N:1 during few scenarios, SBC switchovers to node that was not request honored. When the issue occurred, a switchover will be processed to other node instead of processing to request honored node.

Root Cause: During switchover process SBC was not checking the node which was requested honored. It was switching over to a node that comes first/is available while processing in the SBC.

Steps to Replicate: 

1. Bring up a 2:1 SBC.
2. Try to reboot the both active server.
3. While switchover process delay the up of request honored node and allow the another node to come up first.

The code is modified to check the service id of the request honored node before a switchover.

Workaround: None.

PortFix SBX-106613: The SBC adds duplicate header on new INVITE upon 422 response, when a transparency profile is attached to egress TG.

Impact: The SBC adds a Duplicate Header on a new INVITE after the 422 Response is received.

Root Cause: The SIP Stack adds a semi-known twice (Header is unknown to SIP Parser but configured in transparency Profile) in this 422 scenario.

Steps to Replicate: 

1. Header has to be unknown to the parser.
2. The same header name needs to configured in the Transparency profile.

The code is modified to Skip formatting the second instance of same header based on header type.

Workaround: SMM can be added to remove the duplicate header.

PortFix SBX-110246: The OAM and SBC nodes would not start after revert to 9.1 from 10.0.

Impact: The SBC node fails to start after reverting from 10.0 to 9.1.

Root Cause: The start-up failure after the SBC node revert from 10.0.0 to 9.1 is due to incorrect permissions of logs in /var/log/postgresql/ directory.

Steps to Replicate: Perform revert from 10.0.0 to 9.1 on the SBC node and verify the SBC comes up fine post revert.

The code is modified to update the right permissions on files in /var/log/postgresql/ directory after revert.

Workaround:

Portfix SBX-107973: The SBC adding RR and RS attributes twice in the egress INVITE when multiple m lines present in SDP.

Impact: The SBC was adding RR and RS attributes twice in the egress INVITE when multiple m lines present in a SDP.

Root Cause: The SBC is not setting the default value of M lines present in SDP when number of lines is greater than 1 and as a result, the SIPS value is getting added.

Steps to Replicate: Configure these:
set profiles media packetServiceProfile DEFAULT rtcpOptions rtcp enable

set profiles signaling ipSignalingProfile DEFAULT_SIP commonIpAttributes flags sendRTCPBandwidthInfo enable
set profiles signaling ipSignalingProfile DEFAULT_SIP commonIpAttributes flags sendRtcpPortInSdp enable
comm

set addressContext default zone ZONE_EGRESS sipTrunkGroup TG__EGRESS media multipleAudioStreamsSupport enabled
set addressContext default zone ZONE_INGRESS sipTrunkGroup TG_INGRESS media multipleAudioStreamsSupport enabled
comm

An incoming INVITE has multiple Audio line with:
b=RS:250
b=RR:250

The code is modified so that if value is set to default and it is not getting modified, the SIP does not send the RR and RS value twice.

Workaround: When the sdpAttributesSelectiveRelay is enabled, the SBC does not send RR and Rs twice.

Portfix SBX-108008: SBC: Observed MAJOR logs with MegacoSendAmmsResp failure after a switchover during an EVS and T140 -> Mulaw Load.

Impact: There was a MegacoSendAmmsResp: MegacoSendAmmsResp: Failure while encoding the response, and Error code = 197 Major logs observed after a switchover of the SBC during a load run of call setup with audio and text stream.

Root Cause: The text stream is not properly synchronized to the standby node and triggers the h248 message encoding error in reply to a H248 MODIFY command.

Steps to Replicate: Perform failover during call load run with EVS and ToIP on ingress side and the G711U+Baudot on egress side.

The code is modified to address the issue.

Workaround: None.

PortFix SBX-107142: The SBC VM was unable to power on a post power off procedure.

Impact: The VMware GPU SWe instance does not come up post reboot or GPU is not visible in the instance.

Root Cause: When the SWe instance runs in GPU mode, the persistence mode of the NVIDIA driver is enabled using the nvidia-smi to prevent the GPU state from being unloaded. This is a kernel-level solution and creating problem when the hypervisor is VMware.

Steps to Replicate: 

1. Set sweActiveProfile to use GPU.
2. Reboot the instance after the SBC application is up.

RESULT:

Instance should come up and GPU should be visible in the instance.

The code is modified to address the issue

Workaround: No workaround.

Portfix SBX-105457: An error is thrown on the EMA while configuring the SMM Profile having messageBody criteria with regex.

Impact: An error is thrown on the EMA while configuring the SMM Profile having messageBody criteria with a regex.

Root Cause: The EMA assumes that the Num Match was a mandatory field but it is an optional field (user may enter that value or he may not).

Steps to Replicate: 

1. Log into the EMA.
2. Navigate profile -> signaling -> sip adaptor profile
3. Create the profile that should have a Num match value and after that delete the Num Match value from the CLI.
4. As a result, you cannot see the issue.

The code is modified to make the Num Match field optional.

Workaround: None.

Portfix SBX-107639: The CA CMR with offset 2 and priority LOW is not being processed or rejected.

Impact: When a Channel Aware (CA) Mode CMR for priority LOW and offset 2 was the first CA CMR received in a call, the CMR was not processed.

Root Cause: The present default value of curFecIndOffset in the code was set to 0 that corresponds to offset 2 and priority LOW.

Due to this default value when a CMR for offset 2 and priority low is received as the first CA CMR, it is not getting processed.

Steps to Replicate: Test 1:

1. Enable IR9.2cmr in the SBC.
2. Send ADD request with A1 profile having ch-aw-recv=-1 parameter in T1 termination.
3. Pump pcap with 13.2br with cmr byte of CA-L-O2 followed by CA-H-07

Expected Result:

1. The SBC should accept the ADD request.
2. The SBC should not accept the cmr request. InvalidCMRCount should be incremented

Actual Result:

1. The call is successful.
2. invalidCMRCount gets incremented as per the number of invalid CMRs received

Test 2:

1. Enable IR9.2cmr in the SBC.
2. Send ADD request with A1 profile having ch-aw-recv=0 parameter in T1 termination.
3. Pump pcap with 13.2br with cmr byte of CA-L-O2 followed by CA-L-03

Expected Result:

1. The SBC should accept the ADD request.
2. The SBC should accept the cmr requests. codecModeChangeRxCnt should be incremented to 2.

The code is modified so that the default value does not match any of the valid curFecIndOffset that is 0-7.

Workaround: None

Portfix SBX-108232: SBC: The AddressSanitizer: detected a heap-use-after-free on address 0x61800000a5a8 at pc 0x559e8989c4ac bp 0x7f4411fde290 sp 0x7f4411fde288 READ of size 8 at 0x61800000a5a8 thread T9.

Impact: While the SBC node is shutting down it can access memory after its been freed, this could result in unexpected behaviour and in the worst case a coredump. But would have limited impact as it only happens when shutting down.

Root Cause: During a sbxstop/sbxrestart or switchover because of race-condition, when the SBC is in deactivation the oamNodeRegisterRetry can access already deallocated resource leading to a core dump.

Steps to Replicate: 

1. Setup build with HA SBC using OAM.
2. Perform a sbxrestart/switchover of active instance.

The code is modified to handle this race condition.

Workaround: None.

PortFix SBX-109443: ERROR: The AddressSanitizer: detected heap-use-after-free on address 0x61900412bbce at pc 0x55d0b8c48037 bp 0x7fb50a457b00 sp 0x7fb50a4572b0 READ of size 2 at 0x61900412bbce thread T11.

Impact: The ASAN reported "AddressSanitizer: heap-use-after-free" error for subscribe message received with Proxy-Authorization header having auts parameter.

Ex: Proxy-Authorization: Digest auts*="0x01P+20"

Root Cause: An invalid access of the freed memory occurred in this case. Accessing memory after it is freed can cause unexpected behavior that may result in coredumps.

Steps to Replicate: Send a SUBSCRIBE message received with Proxy-Authorization header having auts parameter.

The code is modified to address the issue.

Workaround: None.

Portfix SBX-109186: [ASAN]: sanitizer.CE_2N_Comp_SamProcess.32180:/localstore/ws/jenkinsbuild/sbxmainasan/marlin/TRM/trmTgPsStat.c:1427:74: runtime error: signed integer overflow: -2147061904 - 552880 cannot be represented in type 'int'

Impact: There was a runtime error: signed integer overflow.

Root Cause: A runtime error occurred due to wrong type casting from ULONG to LONG.

Steps to Replicate: Run the codenomicon INVITE response with outgoing bye codenomicon UAC suite.

The code is modified to address the issue.

Workaround: None.

Portfix SBX-104817: The SBC does not answer the SDP correctly when the remote SDP contains both session and media port state attributes.

Impact: The SBC sets incorrect media port state in the SDP in reply of a MODIFY command when media port state is present in both session level and media level.

Root Cause: The SBC parsed the media port state incorrectly when it is present in both session level and media level, and put them both at media level in the reply SDP.

Steps to Replicate: Create a 3pcc call, then send a re-invite that triggers C3 sending a MODIFY with media port state present in both session level and media level. The SBC should keep them the same session level and media level in the SDP in the reply.

The code is modified when parsing media port state attribute when it is in both session level and media level.

Workaround: None.

Portfix SBX-109209: SBC: Observed MAJOR logs for SIPFE "/localstore/ws/jenkinsbuild/sbxMain/marlin/SIPFE/sipFeSigPortCsv.c, SipFeGetSipSigPortStatisticsGetNextReqMsg, 1111] Another Query in process" on T140 load.

Impact: Observed MAJOR logs for SIPFE "/localstore/ws/jenkinsbuild/sbxMain/marlin/SIPFE/sipFeSigPortCsv.c, SipFeGetSipSigPortStatisticsGetNextReqMsg, 1111] Another Query in process" on T140 load.

Root Cause: The addressContext zone sipSigPortStatistics table is not supported on the SBC, so the code used to return the information was sometimes not returning, causing the error messages above.

Steps to Replicate: 

1. Run the following command: 
   snmpwalk -c admin -v2c <mgmt address of sbx> 1.3.6.1.4.1.2879.2.10.2.121
   Repeat this command 4 times.
2. Verify the following error:
   "/localstore/ws/jenkinsbuild/sbxMain/marlin/SIPFE/sipFeSigPortCsv.c, SipFeGetSipSigPortStatisticsGetNextReqMsg, 1111] Another Query in process" on T140 load
   Does not occur in the DBG log.

The code is modified so SipFeGetSipSigPortStatistics routines return immediately.

Workaround: Do not query that table.

PortFix SBX-109187: [ASAN]: sanitizer.CE_2N_Comp_ScmProcess_0.32472:/localstore/ws/jenkinsbuild/sbxmainasan/marlin/SIPS/sipsParseActions.c:16242:70: runtime error: null pointer passed as argument 2, which is declared to never be NULL.

Impact: The NULL pointer was accessed during the codenomicon test with an ASAN SBC build.

Root Cause: The Codec Attribute has been accessed in the SDP without a proper NULL check.

Steps to Replicate: 

1. Install the ASAN build on the SBC.
2. Run Codenomicon INVITE response with the outgoing Bye suite.

The code is modified to add the defensive NULL check.

Workaround: Not applicable.

Portfix SBX-108619: [ASAN]: /localstore/ws/jenkinsbuild/sbxmainasan/marlin/SIPS/sipsPreParse.c:1742:46: runtime error: signed integer overflow: 2002002002 * 10 cannot be represented in type 'int'.

Impact: A runtime error is seen in the SAM process. When an INVITE is sent out and response code received is very large, we will see the following issue:
Runtime error: signed integer overflow: cannot be represented in type 'int'

Root Cause: When the SIP Response code is very large, there is a signed integer overflow during the processing of the SIP PDU.

Steps to Replicate: An INVITE is sent out to the egress. If the response code received is very large, the issue is seen.

The code is modified so if the SIP response is greater than or equal to max response code, the SBC throws an error.

Workaround: None.

Portfix SBX-108572: [ASAN]: /localstore/ws/jenkinsbuild/sbxmainasan/marlin/SIPS/sipsParseActions.c:12698:76: runtime error: index 20 out of bounds for type 'sip_nameval_str [20]'

Impact: When the PUBLISH message is received with 20 params in the Contact Header, the SBC throws a runtime error: index 20 out of bounds for type 'sip_nameval_str.

Root Cause: The param check for boundary condition was missing while parsing the contact header. While it was checking the params, the number of params should be less than 20, but the condition to handle number of params is not specified.

Steps to Replicate: Send a PUBLISH message with 20 or more params in the Contact header.

The code is modified so if the number of params is 20, the SBC should throw a parse error.

Workaround: None.

PortFix SBX-108411 : [ASAN]: sanitizer.CE_2N_Comp_ScmProcess_2.30828:==CE_2N_Comp_ScmProcess_2==30828==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x61b00008dc88 at pc 0x5621c53c6946 bp 0x7f9854100bc0 sp 0x7f9854100bb8.

Impact: While running a INVITE CANCEL Proxy Call, the UAS observed a Address Sanitizer leak in SCM process and the SBC services stopped.

Root Cause: Issue in parsing string when string is blank and very large that lead to a heap buffer overflow.
Example: SIP/2.0/UDP 10.xx.xx.xx:7009;sigcomp-id=" LWS "

When the token length becomes large 100 say (which is the LWS length) and this token length point outside the PDU, we will see this issue.

Steps to Replicate: Send an INVITE with a VIA header as the last header.

The code is modified to use temptokLen instead of the tokLen. As a result, the tokLen does not reach outside the PDU.

Workaround: The VIA header should not be last header, and the empty token string should be small.

Portfix SBX-109873: The SBC includes the "text port = 0" in its response for the re-INVITE to insert the text at the mid call.

Impact: The SBC rejects the t140 stream in handling a MODIFY command that changes the audio codec from PCMU to AMR-WB and a t140 stream, with "adaptive codec" enabled.

Root Cause: The SBC delays the process of MODIFY command when "adaptive codec change" is enabled, and as a result the MODIFY reply is sent back to C3 without t140 stream media resource allocation (IP and RTP port, etc).

Steps to Replicate: With an adaptive codec change enabled on a VMG, create a 3pcc call from EVS + t140 — PCMU + t140. Then MODIFY T2 side to AMR-WB + t140 and the SBC should reply with a valid port number for t140 stream.

The code is modified to address the issue.

Workaround: None.

PortFix SBX-108410: [ASAN]: sanitizer.CE_2N_Comp_ScmProcess_3.8866:==CE_2N_Comp_ScmProcess_3==8866==ERROR: AddressSanitizer: heap-use-after-free on address 0x6190001c77dd at pc 0x558bcc9ff877 bp 0x7fea305f4e00 sp 0x7fea305f45b0.

Impact: The ASAN reported a "AddressSanitizer: heap-use-after-free" error when Subscribe request having a NULL character in quoted string.

Root Cause: Invalid access of the freed memory occurred. Accessing memory after it is freed can cause unexpected behavior that may result in coredumps.

Steps to Replicate: The codenomicon subscribe-notify suite.

The code is modified to now log a parser error.

Workaround: None.

Portfix SBX-108574: [ASAN]: CE_Node2.log:snprintf buffer too small. Need 327 Have 128 File: /localstore/ws/jenkinsbuild/sbxmainasan/marlin/SIPSG/sipsgLibUtils.c Line: 2524.

Impact: The length of the destination buffer was smaller than the source buffer while calling the snprintf function that why buffer too small error was seen.

Root Cause: The destination buffer size was smaller than the source buffer while calling snprintf.

Steps to Replicate: Run publish requisition codenomicon test suite. This error should not come.

The code is modified to upper limit of source buffer size.

Workaround: None.

PortFix CHOR-7443: Call failures are observed with a 503 error response due to UXPAD process utilization spiked to 20% more that causes overall CPU usage of media container reaches above 90% utilization.

Impact: CPU congestion was observed for media core in 2 VCPU scenarios, leading to call failures.

Root Cause: There was a scheduling issue between NP and UXPAD running on the same core that was causing random spikes of processes causing media congestion.

Steps to Replicate: 

1. Create a 2 VCPU scenario in public cloud and run transcode heavy load.
   (This issue was seen specifically in container environment where we do not have much control on performance tuning parameters).
2. Run the load for 2-3 days.
3. Check if there are any call failures due to media core CPU congestion during the execution.

The code is modified to adjust the process priority of NP and UXPAD to have better scheduling and avoid spikes of processing.

Workaround: None.

The *XrmAsyncCmdErrHdlr: ERROR NpMediaIntf cmd 2c gcid.

Impact: The set grace command returned an error code 0xF0 from the NP, which triggered unsolicited call cleanups.

Root Cause: The set grace command can only be performed on a disabled media flow in the NP. But, thet NP had received some set grace commands issued on an already enabled media flow. Unfortunately, we are unable to determine from just the core WHY XRM and NP became out of sync.

Steps to Replicate: The issue is not reproducible. Perform regular regression tests.

The code is modified to set after XRM has sent media flow enable command to NP and is cleared on media flow disable. The XRM checks this flag and only send set grace commands to NP when this flag is clear. If the flag is set, the XRM reads the  media control block in NP and log a MAJOR level debug message to help future debugging efforts.

Similarly introduced a set of new flags in the BRES structure for RTCP Gen enable/disable commands sent to NP. The BRM checks the new flags and only enables RTCP Gen in NP when the flag is clear. If the flag is set, BRM reads the rtcpGen control block in NP and log a MAJOR level debug message to help future debugging efforts.

Workaround: No workaround.

The FIPS mode was broken on the AWS SBC.

Impact: After enabling the FIPS mode on the AWS SBC or OpenStack, the SBC functions such as 'certificate import' do not work.

Root Cause: While enabling FIPS mode, the FIPS flag is set in sbx.conf. But on Openstack/AWS deployments, the sbx.conf gets reset on the reboot causing a reset of the FIPS flag.

Steps to Verify Fix: 

1. Enable fipsMode from CLI.
2. After a reboot, check /opt/sonus/conf/sbx.conf or try a certificate import from the CLI.

The fipsMode in sbx.conf is set to 'enabled'.

The code is modified to retain FIPS flag in sbx.conf on reboot.

Workaround: After enabling the FIPS mode, set fipsMode=enabled in /opt/sonus/conf/sbx.conf.reconfigHa.orig and reboot the SBC.

After a reboot check, /opt/sonus/conf/sbx.conf

OOD PUBLISH vs. OOD INFO: There are different NRM triggers/congestion profiling.

Impact: OOD PUBLISH vs. OOD INFO: There are different NRM triggers used for congestion profiling.

Root Cause: Some of the OOD message types were not being included for triggering NRM congestion.

Steps to Replicate: Run with carious OOD PUBLISH and INFO messages that should trigger NRM congestion and calculate CPS resource appropriately.

The code is modified to trigger the NRM congestion by including all OOD message types.

Workaround: There is no workaround.

Forwarding info was redirecting info for a customer.

Impact: When testing with JJ90.30 call flows and sending in the following parameters, the RDI (redirection information) parameter created from the history header information had the redirecting indicator field set to “call diverted, all redirection information presentation restricted” even though there was no privacy parameter information in the history header.

Root Cause: The SBC interworking code was following the 3GPP 29.163 specification table 7.4.6.3.2.3 that allows the standard SIP privacy header information to be used when setting the RDI parameter information. Due to the setting RDI parameter information "id", it resulted in the redirecting indicator parameter value of “call diverted. All redirection information presentation restricted” instead of it being set to "call diverted".

Steps to Replicate: Send an INVITE to the SBC, with parameters similar to those in the impact statement and check in the policy request that the RDI redirecting indicator field is set to "call diverted". The SIP trunk group variant needs to be set to TTC and the signaling acceptHistoryInfo control needs to be set to enabled.

The code is modified to not consider the SIP Privacy header value when determining the RDI parameter redirecting indicator field value if the SIP trunk group variant is set to TTC.

Workaround: None.

Increased the healthcheck interval.

Impact: Core dumps are occasionally seen in the field due to the health check timer expiring when the process gets stuck.

Root Cause: In SWe environments, disk/CPU timing issues can lead to the process slowing down and hitting these health checks.

Steps to Replicate: This issue was only reproduced using debug code.

The code is modified to be more forgiving to cope with short spikes. The health check ping interval is now 2 seconds and needs to have 15 consecutive non responses in order for the process to be declared deadlocked and a coredump initiated to recover.

Workaround: None.

TOD Routing broken for non-ALL timeRangeProfiles

Impact: Time of Day Routing is broken for any configured time range profile other than the default ALL profile

Root Cause: The time of the day values are stored in the DB as hexadecimal octets and A to F digits are stored as lower case.
While matching the configured data, the stored values are compared against upper case A to F digits, thus the result of this match always failed.

Steps to Replicate: Set timeRangeProfile other than ALL to test.
set global callRouting route trunkGroup INTERNAL_IPTG99 VSBCSYSTEM standard Sonus_NULL 1 all all TEST0001 none Sonus_NULL routingLabel TO_EXTERNAL_TG99

Note, the TRP TEST0001 is defined as below, and is set to match all days and all times:

set profiles callRouting timeRangeProfile TEST0001 entry 7 timeZone psxLocal

set profiles callRouting timeRangeProfile TEST0001 entry 7 dayMatching dayOfWeek monday,tuesday,wednesday,thursday,friday,saturday,sunday

set profiles callRouting timeRangeProfile TEST0001 entry 7 dayMatching holidays disable

set profiles callRouting timeRangeProfile TEST0001 entry 7 dayMatching specialDays range none

set profiles callRouting timeRangeProfile TEST0001 entry 7 timeMatching range all

Code is updated in the matching function to check for both lower case and upper case hexadecimal digits.

Workaround: No workaround

HA setup on ASAN is not coming up after call config - pathcheck & SMM

Impact: While testing with the following configuration the code could potentially read of the end of an internal memory block while printing a debug log. In the worst case scenario this could result in an SCM coredump.
set addressContext default zone <zonename> sipTrunkGroup <trunk group> signaling messageManipulation smmProfileExecution fixedOrder

Root Cause: The internal structure was not always getting initialized correctly prior to trying to print the contents and this led to the code reading pass the end of the memory block.

Steps to Replicate: Configure SMM with the fixedOrder configuration and make some calls. This issue was highlighted while running with ASAN images in the engineering lab.

The code has been updated to correctly initialise the internal structure and avoid reading past the end of a memory block.

Workaround: Avoid using the "signaling messageManipulation smmProfileExecution fixedOrder" configuration if possible.

Memory leaks are observing on ASAN build when SMM is configured

Impact: While assigning SMM or flexible policy profiles to the zone configuration small amounts of memory leaked.

Root Cause: The code was allocating memory in order to read information from CDB as part of processing the assignment and the memory did not get freed up at the end of processing.

Steps to Replicate: Make SMM configuration changes at the zone level.

The code has been updated to correctly the free the memory.

Workaround: None.

PortFix SBX-108998 to ##9.2.2## - concise view config file is not getting created through REST API.

Impact: concise view config file is not getting created through REST API.

Root Cause: Backend API was unable to read username from curl command to append in exportConfig.sh. which was causing the issue.

Steps to Replicate: 

Export the concise config file using
curl -k -i --user admin:Sonus@123 -X GET 'https://$device_ip:444/pm/api/ImportExport/exportConciseStart&#39;
To check the file name, use the below REST API
curl -k -i --user admin:Sonus@123 -X GET 'https://$device_ip:444/pm/api/ImportExport/getStatus

Modified code to read username from curl command and added in exportConfig.sh to generate the export directory and exportStatus file.

Workaround: Need to update ImportExport.php.

Portfix SBX-110640 to ## 9.2.2 ## - PC 2.0: CDC config not replicated to standby during SWO when there are no active calls.

Impact: In a PC2.0 setup with HA, when a switch over is performed without LI calls then the interception stops working on the new active.

Root Cause: The standby SBC is not processing CDC config for realm2hash. Realm2hash is only created if SBC has active LI calls at the time of transition from Active to standby. In case when SBC don't have Active call realm2hash is not created on transition from Active to standby.

Steps to Replicate: 

1. Configure SBC for LI PC2 flavor call.
2. Disabled the peer and realm.
3. Enable peer and realm.
4. Check the realm2hash entry on current active.
5. Perform switch over from active to standby.
6. Check the realm2hash entry on current active.

Standby code is updated to process CDC config and create realm on standby.

Workaround: Disable/enable CDC Diameter Realm Route and Peer on the new-active.

Portfix SBX-111162:  The SBC fails to defragment/assemble fragmented IPSEC ESP packets received from the customer.

Impact: The SBC drops an in-coming 1500B IP fragments sent through an IPsec tunnel, where the ESP packet is also fragmented into approximately equal-sized IP fragment packets. This complex IP frag-IPsec-IP frag problem primarily affects large SIP INVITE packets in certain network designs.

Root Cause: This problem affects large packets sent through IPsec gateways that (1) do no reassemble received IP fragmented packets before sending them through the IPsec tunnel (IP fragments themselves are sent through the tunnel), and (2) fragment resulting ESP packets into approximately equal-sized packets instead of a 1500B and 72B fragments.

Steps to Replicate: Test should send SIP packets → 1500B to SBC over an IPsec tunnel through a device/devices that:

1. Fragments the SIP packet into 1500B + small IP fragments.
2. Sends the IP fragments through the IPsec tunnel.
3. Fragments (again) the larger ESP packet into approximately equal-sized IP fragment packets.

The code is modified to reassemble IP fragments up to 1580B divided any way into a single internal packet buffer, which the IPsec decryption code and subsequent IP frag reassembly code can handle.

Workaround: Two workarounds are possible:

1. Terminate the IPsec tunnel on a router in front of the SBC instead of directly on the SBC.
2. Do not send the IP fragments through an IPsec tunnel terminated on the SBC. Instead, reassemble IP packets before sending them through the tunnel towards the SBC, so that there is only one level of IP fragmentation (of the ESP packet itself).