SBC Core 09.02.05R009 Release Notes

Verify system and databases are fully in sync prior to Live Software Upgrade (LSWU). Applies to all SBC platforms (HW, SWe, Cloud) except the SBCs deployed in a Distributed SBC (D-SBC) architecture.

SBX-122469

(10.1.3)

An EG UAT SWe SBC switchover occurs when SRTP and SRTCP calls are made.

Impact: SWe_NP process cores and a switchover occurs when the SBC processes certain types of malformed RTCP packets for SDES CNAME replacement with RTCP passthrough.

Root Cause: A defect in the RTCP SDES parsing code resulted in memory corruption.

Steps to Replicate: 

1. Configure the SBC for passthrough SRTP calls.
2. Set RTCP mode to passthrough.
3. Enable the ssrcRandomizeForSrtp flag in PSP.
4. Make a call and send in various valid and invalid format RTCP packets for the call, including an RTCP packet with a single-chunk SDES with non-zero pad bytes at the end.

The code is modified to alter the RTCP SDES CNAME replacement code in order to correct (if possible) or drop various types of malformed RTCP packets.

Workaround: Disable CNAME replacement and ssrcRandomizeForSrtp features in PSP (if possible).

An SCM process core occurred on the Server.

Impact: An SCM process cores due to a double free of an internal structure.

Root Cause: An SCM cores due to a double free of an internal structure.

The structure is in an array of these structures. The array is a fixed size. When attempting to add a new entry to this array, we consolidate the array by removing some of the free entries.

In the unlikely event that all of the array entries are in use, we remove the oldest entry and consolidate the array by moving the other entries up.

There is a bug in this consolidation code that causes a double free when the entire array is later freed.

Steps to Replicate: The steps are not reproducible.

The code is modified to clear out the pointers in an array entry after it moves up to the previous entry during consolidation.

This ensures that there are no leftover/stale pointers that might cause a double free.

Workaround: None..

SBX-122577

A core dump occurs on a customer SBC with a processAbnormalTermination.

Impact: The CPX cores when deleting a CDR server.

Root Cause: The code was missing a check for element type when processing a deletion in the CDR server.

Steps to Replicate: The steps are not reproducible. 

The code is modified so the element type is processed before deleting.

Workaround: None.

SBX-122225

(10.1.4)

Call trace status getting enabled after a switchover using the EMA.

Impact: Call Trace staying enabled in new active node when switching over, when disabled on the current active.

Root Cause: No notifications were getting sent to the standby SBC after issuing a stop command.

Steps to Replicate: 

1. Enable call trace through the EMA/CLI, and then disable it with a stop command.
2. Perform a switchover and check the status on the new active SBC.
3. Try the same with the Call filter enabled.

Test Result: The call trace state is disabled on the new active.

The code is modified so the correct notification is sent to update the standby node when Call Trace is being disabled.

Workaround: Without the fix, the workaround is to disable call trace using new active node's EMA or CLI on, if it is enabled after switchover.

A customer SBC switchover occurred due to an SCM core.

Impact: The SCM cores when processing a call that is using SIP Recording.

Root Cause: The code is attempting to access a SIP Recorder call block that is already freed. This occurs because of an edge case call scenario where the call block is freed.

Steps to Replicate: This bug was found using the back-trace and through code inspection. The core is happening after the call block gets freed. Attempts to recreate the issue are unsuccessful.

The code is modified to always remove the SIP Recorder call block from the hash table before freeing the call block.

Workaround: None.

The SBC is slow to relay the first register.

Impact: PES process is slow to process a D+ request when no call notification group attached to TG.

Root Cause: While processing the call notification criteria matching, PES processes all the required strings from called and calling number and then it used to check if there is a call notification group attached to the TG. This is done for each and every route. This process previously took up the processing time when there is no call notification group is attached.

Steps to Replicate: Make a basic call and find the difference between the start and end time of PES processing the call. Performance related - may only see difference under a load.

The code is modified to check if the call notification criteria is attached to the TG, and then only process the rest of the required strings.

Workaround: None.

SBX-122304

(10.1.4)

The standby M-SBC did not take over when the active SBC rebooted in an N:1 cluster.

Impact: Switchover did not occur after the SWENP process crashed due to a race condition.

Root Cause: Infrequently, a switchover event may not be sent to the standby in specific case of SWENP process crash on an active SBC.

Steps to Replicate: The steps are not reproducible.

The code is modified to ensure a switchover event is sent and processed by standby on SWENP crash and other switchover scenarios.

Workaround: None.

In the 925R4, a few calls fail with "NrmaAllocResCmd: paramInsert failed" error

Impact: Multi-streams call with large SDP (due to crypto and other attributes) fails if DM and DTLS enabled at the SBC.

Root Cause: Interprocess communication buffer requirements for multi-stream call is higher compared to other call flows. However, at present, higher buffer is allocated only if number of streams in call is six.

Since the number of streams is not six, higher buffer is not allocated resulting in failure.

Steps to Replicate: 

Enable SRTP/DTLS/DM at the SBC.

To re-create the issue: UAC sends INVITE to the SBC with 5 streams (mix of audio/video/application) with crypto and other attributes.

Expected Result: The call is accepted by the SBC and succeeds.

Actual Result: The call is rejected by the SBC.

The code is modified to allocate a higher buffer if the call involves four or more streams.

Workaround: None.

SWE PAID and FROM translation tables not working post 9.2.4R2 upgrade

Impact: The SWE PAID and FROM number translation criteria (cpe type: tgWithCallingNumber) are failing to match after an upgrade or restoration of the backup.

Root Cause: When the backup is restored, the info table of tollfree prefix info is not populated properly with the length of the calling number (call processing element type 3). It is getting populated with the length of the trunk group (call processing element type 1). In call processing, when trying to find the record of the trunk group with the calling number, the SBC uses the length of the calling number and not the length of the Trunk group. The same series of events occurs in the upgrade scenario, so the customer encountered this issue post upgrade.

Steps to Replicate:

1. Take a backup with the existing config.
2. Restore the backup or upgrade to the next release.
3. Test the calls with tgWithCallingNumber as criteria.

The code is modified to populate the toll-free prefix info table properly for restore and upgrade cases.

Workaround: None.

The SBC cores after upgrade to v09.02.05R003

Impact: The SBC PRS Process cores with NP indicating multiple cores failing. Memory dump analysis shows problem in the RFC2833 DTMF digit detection area of the code.

Root Cause: The problem happens when an ICE/DTLS call is established followed by the endpoint sending RFC2833 DTMF digit packets and with DTMF Trigger Profile is enabled. A pointer to store the digit detected in memory is getting reset to zero after the ICE/DTLS call is established.

Steps to Replicate: 

1. Enable DTMF Trigger Profile.
2. Make an ICE/DTLS call.
3. Send an RFC2833 digit packet to the SBC within the ICE/DTLS call.

The code is modified to set the pointer to store digits detected properly after the ICE/DTLS call is established.

Workaround: Disable DTMF Trigger Profile.

The SCM Process cores and switched over

Impact: Direct Media with multiple audio and ICE call may cause the SBC to core.

Root Cause: When receiving multiple audios with ICEs, Ingress is passing only one ICE info of active stream to egress. When the egress leg tries to retrieve the ICE info, it expected to receive two ICE info for two streams. As a result, it reads garbage data which may cause the SBC core.

Steps to Replicate: 

1. Enable Direct Media, ICE support, multipleAudioSupport.
2. A call B with multiple audio streams+ICE.
3. B re-IVITE with multiple audio stream+ICE. The SBC may core.

The code is modified to correct the logic on the ingress leg that passes all ICE info for each stream to the egress leg.

Workaround: Disable ICE.

No Beep Tone issue

Impact: The egress endpoint did not hear a Beep Tone from ingress.

Root Cause: When the ingress leg switched to tone player to play beep tone, the resource chain was rebuilt and NAPT learning was restarted on egress leg with RTP flow mode = rcv-only. Hence, no RTP packets are sent to the egress endpoint.

Steps to Replicate:

1. Make a SRTP call with egress endpoint behind the NAT.
2. Switch to a different media server to play the tone via re-INVITE, and check the audio at egress endpoint.

The code is modified to set RTP flow mode to duplex, if the SBC already learns the remote IP address, when enabling NAPT re-learning process.

Workaround: None.

Transparency profile causes a core dump SBC-GW-GW-SBC

Impact: SCM may core during a GW-GW call while transparently forwarding P-Sig-Info header:

• A call is GW-GW and contains P-Sig-Info
• Transparency Profile on Egress GW contains P-Sig-Info header

Root Cause: The core occurs because the code that is processing the P-Sig-Info header is attempting to access invalid memory.

Steps to Replicate: 

- Call is GW-GW
- INVITE contains P-Sig-Info
- Transparency Profile on Egress GW contains P-Sig-Info header

NOTE: This may not be easily reproducible. This code hasn't changed in a long time - but we haven't seen this issue before. This is probably because most of the time this bug may not cause a core. Sometimes it will simply set the pointers in P-Sig_info to addresses that are incorrect but still within the valid memory space (which will NOT cause a core).

The code is modified to ensure that the internal representation of the P-Sig-Info header points to valid memory.

Workaround: Remove P-Sig-Info from Transparency Profile.

There was an SSN issue with ICE/TLS/SRTP/P2P

Impact: Possible audio issue involving an incorrect SSN when using ICE/TLS/SRTP/P2P.

Root Cause: The SSN value 0 was incorrectly saved in XRES on deactivation for the case when the RID was never enabled in NP while performing NAPT or ICE learning.

Steps to Replicate: Use on hold and transfer signaling to interrupt learning during the early stages of the call to potentially get a random SSN and ROC values, which the far end rejects.

The code is modified after the NAPT or ICE learning of a remote address is completed, ignoreRocSsn is set and causes NP to use the random SSN/ROC values from RID memory block. This fix is a similar fix to SBX-119900. 

Workaround: None.

SBX-123228

(10.1.3)

DTMF Intermittently not working post upgrade.

Impact: Audio loss after DTMF digits.

Root Cause: After ingress received a re-INVITE for a media reconfig, we used a new RID to replace the previous RID. But when the resource chain was re-activated, the new RID was programed SSN = 0 that caused the audio loss.

Steps to Replicate: The steps are not reproducible.

The code is modified to:

1. Assign a valid SSN/ROC for the new RID.
2. Ensure the SSN value is greater than, or equal to, the last saved SSN on the deactivation for anti-replay protection.

Workaround: Enable “SSRC Randomize For Srtp”.

Unnecessary re-INVITE was removing a 2833 message

Impact: A DTMF telephony-event gets dropped in response to a mid-call re-INVITE.

Root Cause: A previous enhancement to support both 8K and 16K DTMF and to transcode 16K, when necessary, contained a defect in the logic that resulted in a call appearing to be a pass-through call.

Steps to Replicate: 

1. Ingress UAC endpoint offers DTMF 8K PT 100 to SBC1 , SBC2 offers DTMF 8K PT 101 to UAS endpoint (SIPP).
2. Egress UAS endpoint(Sipp) answers without DTMF/8000 (DTMF) to SBC 2.
3. SBC 1 answers with DTMF PT 100 to the Ingress UAC endpoint (SIPP).
4. Call is setup as xcoded between SBC 1-SBC2.
5. Egress UAS endpoint re-INVITES to add DTMF PT101 to SBC 2.
6. SBC 1 re-INVITES with PT 100 to Ingress UAC endpoint.

The code is modified to correct the logic that was not correctly identifying a call as transcoded between DTMF payload types 100 and 101.

Workaround: None.

SWe_NP - NP crashed with SIGSEGV

Impact: The SWe_NP crashes due to a SIGSEGV event.

Root Cause: The Segmentation table (a table that points to announcement buffers and has meta data for a call) gets initialized during ANN PLAY command.

Since an ANN play command was never issued for a given call, the segmentation table was never initialized. The state machine in NP allowed the execution of a second disable command to proceed (which ideally should not be allowed) and tried to flush and build an announcement packet. Since the Segmentation table was never initialized, the source buffer pointer was NULL and crashed with SIGSEGV.

Steps to Replicate: The steps are not reproducible.

The code is modified to the flush and disable the code to handle such conditions.

Workaround: None.

An EMA Prefix Profile display issue occurred after upgrading to 9.2.5R1

Impact: In the Prefix Profile Entry, the values for the following parameters (Matching Pattern, Match Start Location, Total Min Digits, Total Max Digits, Digit Type, DM PM Rule, Apply DM Rule and Determine Area) are not displayed for matching pattern starting with '+'.

Root Cause: The URLDecoder was replacing the symbol '+' with blank space ' ' in the method decodeTextFromUI.

Steps to Replicate:

1. Log into EMA as admin user.
2. Go to All -> Profiles -> Digit Parameter Handling -> Prefix Profile -> Entry.
3. Under the Entry list, select any matching pattern starting with '+'.

Values for the following parameters display (Matching Pattern, Match Start Location, Total Min Digits, Total Max Digits, Digit Type, DM PM Rule, Apply DM Rule and Determine Area).

The code is modified to replace the symbol '+' with '%2B' before decoding in the method decodeTextFromUI.

Workaround: None.

A customer SBC failed-over

Impact: For t140<=>baudot interworking, if the t.140 packet with a t.140 block contains a long string with alternating letter and numbers, then it can cause a DSP core dump (Swe_UXPAD). The following string was observed to cause crash "q1q1q2q1q1q221q1q1q1q1q1q2 1q".

The SBC allows maximum length of 36 bytes for a t140 block.

Root Cause: The buffer allocated to convert t140 block to baudot characters was insufficient for a long character string with frequent transition between figure and letter characters.

Baudot character sets have FIGURE and LETTER modes and to switch between the two, a mode switch character has to be inserted. A long character string with frequent transitions resulted in buffer overflow and caused crash.

Steps to Replicate:

1. Setup a AMRWB+t140 =>g711 (baudot) call.
2. Send a t140 packet with a t140block consisting of following characters. This is the character string that was observed in crash.
   "q1q1q2q1q1q221q1q1q1q1q1q2 1q"

The code is modified to increase the buffer size to the maximum possible baudot string size (i.e. 72 baudot characters).

Workaround: None.

SO-SBC (4+1) sends out-of-order 200-OK (SUB) and NOTIFY

Impact: The SBC is processing the NOTIFY ahead of the 200 OK for SUBSCRIBE message in certain scenarios where there is no delay between the 200 OK and NOTIFY messages arriving at the SBC.

Root Cause: The SBC uses the two digits before the B in the following branch prefix = z9hG4bK18B. In this example, '18' identifies the SCM process which handled the 200 OK response. Due to a bug in the code, the SBC was only looking at one digit, and so the SCM process could not be determined. The SBC had to hold up the 200 OK response until it did a TRM query to determine the SCM process that handled the message.

In the meantime, the NOTIFY arrived and it was processed immediately based on the same digits in the tag parameter i.e. tag=gK18e.

Steps to Replicate: 

Continually run a call flow where the SBC relays SUBSCRIBE messages and the remote party sends back 200 OK SUBSCRIBE and NOTIFY at the same time.

Check that the SBC forwards the messages in the same order; i.e., a 200 OK SUBSCRIBE first, and then a NOTIFY to the A-party.

The code is modified to use the two digits of the branch parameter to route the 200 OK for SUBSCRIBE so there is no delay in processing and the messages are processed in order.

Workaround: None.

SBX-121606

(10.1.4)

Allow user-config-import on the OAM

Impact: The user-config-import command is unavailable on an OAM node.

Root Cause: This limitation is being removed as part of an enhancement request.

Steps to Replicate:

1. Run user-config-export on one OAM node.
2. Update the configuration to fit with the new OAM node.
3. Import the configuration into the new OAM node.
   

The code is modified to allow for the user-config-import command on an OAM node. This should only be used for greenfield deployment of a new cluster using the configuration from an existing golden configuration of another cluster.

Workaround: Back up the configuration in CLI format and then edit the CLI to use the correct import order into the new cluster. Update the new cluster specific configuration in the CLI file and then source it in.

SBX-122159

(10.1.4)

The ICE process does not immediately respond to a call audit 

Impact: The call cleanup process always takes the full five seconds to complete, even when all the expected processes respond.

Root Cause: The five-second timer for a process starts, but a response never occurs. As a result, the cleanup process is delayed by five minutes.

Steps to Replicate:

1. Set up a call with RTP inactivity of 40 seconds with a trap and disconnect. This should result in approximately 1,000 calls with a 40-second gap between calls.
2. Attach a LRBT and duration of call should be around 42-44 seconds.
3. Search for repeated occurrences of logs "not known by NRM".

The code is modified to reduce the amount of unnecessary calls in the ICE process. 

Workaround: None.

Rework on the SBX-115152 logic.

Impact: Additional logic change for SBX-118384/SBX-115152 resolved originally in 9.2.5R0.

Root Cause: In certain call scenarios, the call control block gets reset on the egress side. This results in the SBC deleting a data pointer to one call leg, thus affecting the SBX-115152/SBX-118384 9.2.5R0 bug fix..

Steps to Replicate: 

1. Establish a SIP-SIP call with G711 U law.
   1. Enable "Minimize Relaying Of Media Changes From Other Call Leg" flag in Ingress IPSP.
   2. Enable "Unknown Header" transparency in IPSPs.
2. After establishing the session, send a re-INVITE from the Egress peer with G711U law for call hold (a=sendonly).
3. Send BYE from the Ingress peer with Unknown Header.
4. Verify if the unknown header in BYE message is sent transparently to the other leg.

The code is modified to add the call control block based on the call direction.

Workaround: None.

The GSX generates a STOP CDR while corresponding with ATTEMPT CDR and not STOP record in the SBC

Impact: In a SBX-GSX GW call, the GSX generates a STOP CDR while the SBC generates an ATTEMPT CDR instead of a STOP CDR.

Root Cause: This issue is not fully understood and cannot be recreated.

Steps to Replicate: Not identified.

The code is modified to add enhanced logging to collect more info.

Workaround: None.

The SBC 5400 is stuck in a LSWU after a switchover or restart

Impact: Sync does not complete after a switchover or restart when using Call Notify functionality.

Root Cause: Sync does not complete after a switchover or restart when using Call Notify functionality because the standby code detects a duplicate entry in the Call Notify array and returns an error.

Steps to Replicate: 

1. Configure the SBC to send unsolicited call notifications to Application Servers.
2. Run a load test with Call Notify functionality enabled and force a switchover.
3. "set addressContext <ac name> zone <zone name> sipTrunkGroup <tg name> services sipCallNotificationMetadataProfile <meta data profile name>"
   Note: This is a race condition that will only occur if Notifications are being processed during a switchover or sync.

For more details on this configuration, see Configuring the SBC to Send Unsolicited Call Notifications to Application Servers.

The code is modified to replace the existing entry in the Call Notify array with the new one when a duplicate is detected.

Workaround: None.

SBX-123550

(10.1.4)

During a LSWU, audio was lost for incoming calls, but was present for outgoing calls

Impact: When ssrcRandomizeForSrtp is enabled, a one-way audio issue occurs after the SBC switches over to the standby node

Root Cause: If ssrcRandomizeForSrtp is enabled, the NRMA generates an SSRC for the SRTP leg. The SSRC is sent to the NP when enabling the associated RID. In a hot standby setup, that SSRC is mirrored to standby, but it is not sent to the NP when enabling the same RID on the standby. Instead, the NP adapts the SSRC from the incoming RTP stream or the DSP's SSRC. Once the standby node becomes active, the NP changes the SSRC.

If the endpoint sees a new SSRC, it starts decrypting using ROC=0. However, the NP was encrypting on send on the SRTP leg with ROC=2 that prevented the endpoint from decrypting, which resulted in one-way audio.

Steps to Replicate: 

1. Run an HA set up.
2. Enable the ssrcRandomizeForSrtp.

3. Make a long duration RTP to SRTP transcode call, and then check audio.

4. Switch over to the standby node after the call has been up for over 25 minutes.

5. Revert back, and then check audio.

6. Repeat above step 1 to 5 for a long duration pass-through call.

7. Repeat above step 1 to 5 for a short duration transcode call.

8. Repeat above step 1 to 5 for a short duration pass-through call.

The code is modified to send NRMA generated SSRC to NP when enabling the RID on standby node so that NP does not adapt incoming RTP stream's SSRC.

Workaround: None.

Special characters incorrectly display using the "View CLI" option

Impact: In the SMM GUI, a couple of special characters incorrectly display in the "View CLI" option.

Root Cause: The '<' character is sent as '&amp' instead of '&lt;' and the \\r\\n => double backslash escaped properly but display as a single backslash in HTML.

Steps to Replicate:

1. Log into the EMA and navigate to the SMM screen.
2. Select the profile that includes the characters < \\.121394
3. Click the View CLI button for the selected profile.

The CLI commands display as expected.

The code is modified to replace those occurrences with correct responses to address the issue.

Workaround: None.

Multiple PRS process cores occurred on the server

Impact: The PRS process cored due to a XRM task failed health check.

Root Cause: The sigPortList in the XRM got corrupted. Numerous freed XRESs were left in it. When trying to activate a new signaling port, the XRM task got stuck in the loop and failed a health check.

Steps to Replicate: The steps are not reproducible. 

The code is modified to properly free already allocated XRES for other pool(s) during a signaling port allocation routine

Workaround: None. 

A call transfer (REFER with Replaces) fails with an OA status 0x54

Impact: An attended call transfer failed due to a race condition when the transferor sends an off-hold INVITE and REFER with Replaces in rapid succession.

Root Cause: While the SBC is in the process of bridging a transferee and transfer target, the off-hold INVITE from the transferor triggered a new modify request towards transferee, which caused an offer-answer timeout internally due to previous modify still being processed that leads to a call failure (OA status 0x54).

Steps to Replicate:

1. A calls B and the call is connected.
2. B puts A on hold.
3. B (you can also assume B' as User C) calls D and the call is connected.
4. B sends an off-hold INVITE to the SBC.
5. C sends REFER with Replaces of B to the SBC.

The code is modified so that the internal offer-answer timeout is handled to avoid call transfer failure.

Workaround: None.

A unique tag is created for a Notify when the 200 OK was not sent yet in response to a Subscribe

Impact: When a Notify received before the initial 200 OK response to Subscribe, the SBC may send an invalid tag in 200 OK Subscribe.

Root Cause: After receiving a Notify from Egress, the SBC is using egress service group id as part of tag in 200 OK Subscribe for sending to the ingress.

Steps to Replicate: Run the following configuration:

Subscribe ->Subscribe
                 <-Notify
                 <-200 Subscribe

The code is modified to relay a 200k Subscribe to the Ingress service group id.

Workaround: None.

Receiving a syslog spam about ACL Stats from the SBC1b.

Impact: /var/log/syslog fills up with cpsi log entries about unable to read ACL statistics.

Root Cause: A problem was found in CPS (Common Platform Service) library retrieving the next resource. CPS keeps a list of allocated resources and the CpsXyzGetNext API is used to grab the next entry in the list. The next resource information was stored in common shared memory and was not thread safe. If multiple threads invoked the same API simultaneously, the result would be unpredictable.

Steps to Replicate: The steps are not reproducible.

The code is modified to add a limit to the loop count for the number of allocated resources to ensure the caller of the function passes in a CpsXyzGetNext API storage location to make it thread-safe.

Workaround: None. If the kernel syslog gets filled, you must reboot Linux to stop the problem.

The LWRESD process keeps restarting causing the SBC app to not come up.

Impact: The LWRESD Process continues to restart as the interface used by it for ENUM/DNS signaling was down.

Root Cause: When a SIP SIG port is selected as the signaling interface for LWRESD, by design the SBC uses the query-source-address parameter in the slwresd.conf file. Upon getting this parameter, the bind stack tries to create a socket on that IP and port. If it is unable to create the socket, the bind stack exits abnormally.

Example sysdump syslog failure indication:

<27> 1 2022-08-30T09:45:05.191451-04:00 vsbcSystem lwresd 29904 - - could not get query source dispatcher (10.0.4.248#988)
<26> 1 2022-08-30T09:45:05.191769-04:00 vsbcSystem lwresd 29904 - - loading configuration: address not available
<26> 1 2022-08-30T09:45:05.191962-04:00 vsbcSystem lwresd 29904 - - exiting (due to fatal error)
<29> 1 2022-08-30T09:45:05.195760-04:00 vsbcSystem lwresd 29904 - - running

Steps to Replicate: 

1. Set up the LWRESD profile to use the SIP signaling interface.
2. Bring the SIP signaling interface mode or state to outOfService/disabled state.
3. Restart the SBC application.

The SBC app does not come up in service as the LWRESD process keep on restarting.

The code is modified to generate the default slwresd conf file if both the SIP signaling port type is 'SIP Signaling IP' and if the signaling port is disabled, since LWRESD cannot perform DNS signaling. The process generates the slwresd.conf based on the configuration when the interface is changed to the in-service state using the CLI.

The LWRESD Profile now maintains two different bit masks for the interface mode and state. Consequently, after upgrading to an SBC version that includes this fix, use the following steps if the LWRESD Profile type is set to signaling IP:

1. Set both the state and mode of the sipSigPort that is configured in the LWRESD profile to disable and outOfService respectively.
2. Set both the state and mode of the sipSigPort that is configured in the LWRESD profile to enable and inService respectively.

Note: These steps are only required for an upgrade procedure if the signaling type is 'signalingIP' for the LWRESD Profile.

Workaround: None.

SBX-120330

(11.0.0)

UX_PAD is dumping core while running evrc to g729 load.

Impact: A UXPAD Crash observed while running EVRC calls on GPU.

Root Cause: As an oversight, the GPU software does not initialize all GPU EVRC decoder context variables, including the variable used as an index to an array. This led to an illegal memory access occurrence.

Steps to Replicate: The steps are not reproducible. 

The code is modified so the decoder context variables are initialized to their default values.

Workaround: None.

SBX-115363

(11.10)

AWS: eth1 and eth2 subnets are getting interchanged in an HFE instance.

Impact: The AWS HFE Node can sometimes have the wrong IPs assigned to the interface.

Root Cause: The AMI image incorrectly sets the interface configuration.

Steps to Replicate: 

1. Delete the default route for eth2.
2. Re-run the HFE script to ensure it completes correctly.

The code is modified to:

1. Correct the 'Debug SBC SWe in AWS' page on how to correctly reconfigure the interfaces.
2. Handle finding the gateway IP, when the default routes are not available for the interface.

Workaround: None.

Dead air after call is resumed from hold

Impact: SSN/ROC reset after hold that caused one-way audio.

Root Cause: In order to prevent reading random values of SSN/ROC in the RID’s memory block, XRM has cleared the remote address saved in XRES's security context. But that change should only apply to the case when the call flow uses a new RID after call modification.

In this call flow, the same RID was used after hold. But the SSN/ROC got reset unnecessarily after hold.

Steps to Replicate: 

1. Enable NAPT learning on egress SRTP call leg.
2. There is no S-SRC change.
3. Run call hold and resume test.

The code is modified to introduce a new flag in XRES's security context. The flag is only set to "true" when XRM detects a new RID. Then when NAPT learning times out, XRM checks this flag to determine whether or not to clear the XRES-saved remote address.

Workaround: None.

GW-GW calls are failing after an upgrade from 9.2.3R3 to 9.2.5R2. The SBC stops processing the call after parsing the MCS EST_CONF message from the egress GSX.

Impact: Calls between the SBC and GSX may cause an SCM core dump.

Root Cause: A bug was introduced in an internal function. CpcMsgInfoCreateCopy(), which copies the CPC Optional Parameters that are used to pass information about a call between processes and between systems.

This bug causes us to calculate the offset to the next parameter incorrectly. When the pointer is incorrect, the code that validates the CPC Optional Parameter will detect an error and cause a core dump.

Steps to Replicate: This issue was found by code inspection. The steps are not reproducible.

The code is modified so that it now calculates the offset to the next parameter correctly.

Workaround: None.

SBX-113502

(10.1.0)

CE_2N_Comp_ScmP SYS_ERR generated

Impact: Calls between the SBC and GSX may cause an SYS_ERR and not process the internal parameter content correctly.

Root Cause: A bug was introduced in an internal function - CpcMsgInfoCreateCopy(), which copies the CPC Optional Parameters that are used to pass information about a call between processes and between systems.

This bug causes us to calculate the offset to the next parameter incorrectly. When the pointer is incorrect, the code that validates the CPC Optional Parameter will detect an error and cause a core dump.

Steps to Replicate: This issue was found by code inspection.

The code is modified so that it now calculates the offset to the next parameter correctly.

Workaround: None.

There was a core and failover on the SCM Process coredump and the SBC failed over.

Impact: The SCM process cores after an attempt to free a memory block that is already free.

Root Cause: A memory block was freed but did not clear the pointer to that freed memory block that is stored in the main Call Block.

As a result, the app does not know that this memory block has already been freed. If the app tries to free it a second time, a core is caused.

Steps to Replicate: This bug was found by code inspection, and is therefore not reproducible.

The code is modified to clear the pointer to the freed memory block that is stored in the main Call Block.

Workaround: None.

A memory leak occurred in an Advanced SMM dialog variable.

Impact: A memory leak occurred in an Advanced SMM dialog variable.

Root Cause: When the egress leg responds with 18x, the SBC fails to update the call control block with new dialog variable ID. As a result of the call teardown before 2xx, the SBC fails to delete the memory of SMM dialog variable.

Steps to Replicate: 

1. Configure an Advanced SMM with dialog variables and dialog transparency.
2. Run an A call.
3. B reply 1xx and wait for the timeout for the SBC to tear down the call.

The code is modified to always update the call control block with new dialog variable ID when received the responds from peer.

Workaround: Disable Advanced SMM.

SBX-120826

(10.1.3)

ASAN - run-time error " runtime error: load of value 32524, which is not a valid value for type 'SIP_TRANSPORT_ENUM' in sanitizer logs

Impact: ASAN reported "runtime error" error while reading a value related to the transport type while processing a 407 response to a REGISTER message.

Root Cause: The code was reading from uninitialised memory while processing the 407 message, which led to reading an unexpected value.

Steps to Replicate: Only reproducible while running failed registrations in Ribbon lab using ASAN build.

The code is modified to ensure data is initialised correctly to avoid the problem.

Workaround: None.

The SBC sent SYN to a client port instead of port 5060

Impact: The SBC tried to reopen TCP connection to the client for an incoming call and with the SBC in role of server side using TCP and the connection is already closed. The SBC should not try to reopen connection.

Root Cause: The same logic will be applied for the SBC TCP connection as a client

Steps to Replicate: 

1. User A (TCP) calls to user B.
2. User A disconnects/closes the TCP connection.

The packet capture shows the SBC trying to reopen the connection in order to send a status message.

The code is modified to prevent the SBC from reopening a connection for an SBC TCP connection as a server.

Workaround: None.

SBX-121620

(11.1.0)

The SBC is not adding country code +81 in Request URI while sending out INVITE towards egress peer in E2E call flow.

Impact: The SBC is not adding country code in Request URI of INVITE.

Root Cause: The SBC is not globalizing the number by assuming that calledURI is already globalized.

Steps to Replicate:

1. Set the country code as 81 in TG.
2. Run a basic call.

Expected Result: Check whether the SBC is adding the country code in RURI, the SBC does add country code for RURI.

The code is modified to check whether the calledURI globalized or not. If it is not globalized, the SBC does globalize it and adds a country code.

Workaround: None.

SBX-121633

(10.1.4)

Missing UI control for Level 4 Trace Messages configuration

Impact:

1. Login to EMA.
2. Go to All -> Global -> Call Trace.
3. Level 4 Trace Messages configuration is missing in Call Trace Status and Settings.

Root Cause: Level 4 Trace Message has been added as an Enhancement in EMA.

Steps to Replicate:

1. Login to EMA.
2. Go to All -> Global -> Call Trace.
3. Go to Call Trace Status and Settings.
4. Choose All Messages or Received Invite under Level4 Trace Messages.

The code is modified to add Level 4 Trace Message configuration:

• allMessages - Level 4 call traces contain all matching messages.
• rcvdinvite - Level 4 call traces contain only matching received INVITE messages.

Workaround: None.

SBX-122013

(10.1.3)

RCS SBC - Dropping ingress TCP-segmented MSRP packets (parsing error)

Impact: Dropping ingress TCP-segmented MSRP packets (parsing error).

Root Cause: The transaction ID was getting mismatched due to low buffer ENUM used for the transaction ID copy, truncating the transaction ID that is later compare to match the End of the Packet.

Steps to Replicate: Test the MSRP message having Transaction ID of length 32 characters.

The code is modified to increase the buffer size to 4 bytes to compensate for the full length of the transaction ID that can be at maximum length of 32.

Workaround: None.

SBX-122015

(11.1.0)

The P-charging vector header has ipx3.xxx.com.1 instead of ipx3.xxx.com.2

Impact: On the egress Leg, the SBC is sending wrong PCV values in the PCV Header.

Root Cause: On the egress leg, the SBC was supposed to add PCV header. However, the SBC is applying transparency logic to PCV header and as a result, the SBC send wrong values in PCV header.

Steps to Replicate: 

An INVITE is received with PCV header

P-Charging-Vector: icid-value=a9fbd640-30e4-1032-00-00-00-10-6b-03-18-01;icid-generated-at=10.xx.yy.zz;orig-ioi=operatorb.com;transit-ioi="operator_A.1"

The sendPCVHeader is enabled on egress leg and PCV transparency is enabled. The SBC sends the generated PCV header and not transparently.

The code is modified so when the SBC adds a PCV header, the transparency is not applied again.

Workaround: None.

SBC manager path check URL character violation

Impact: An SBC Configuration Manager path check creates a URL character violation.

Root Cause: Since the values are not encoded, the ^ symbol creates an issue.

Steps to Replicate: 

1. Go to SBC Configuration Manager in the EMA.
2. Navigate to pathCheck screen.
3. Select addressContext -> Zone -> IP Peer values.
4. Scroll down to Commands section.
5. Select value from the dropdown and click Select.
   The SBC manager starts accepting requests from EMS in the URL.

The code is modified to encode the values of form elements, which resolves the URL character violation.

Workaround: None.

SBX-122178 

(11.1.0)

The SBC is not rejecting the INVITE received over unsupported transport protocol

Impact: The SBC is not rejecting the INVITE received over unsupported transport protocol.

Root Cause: The SBC returns a failure in an early stage that causes the SBC to select the default sipSigPort. Due to this, the SBC is further processing the INVITE.

Steps to Replicate: 

1. Set the allowed transport on sipSigPort as TCP.
2. Send registration over TCP and INVITE over UDP from the same script.

The code is modified to not select the default sipSigPort.

Workaround: None.

SBX-120767

(10.1.4)

RESTAPI having 'set' operation limitation when using POST API commands.

Impact: RESTAPI having 'set' operation limitation when using POST API commands.

Root Cause: The validation point for the /META:metadata table was counting all changes in the current transaction, not just changes to the META:metadata table. This caused the limit of metadata table transactions of 1000 to be exceeded.

Steps to Replicate: 

1. Create a rest request that creates a sipSigPort and three sipTrunkGroups.
2. Execute that command.
3. Verify that all items are correctly added to the confd database.

The code is modified to only count changes to the META:metadata table.

Workaround: Split the REST request into smaller requests to avoid this issue.

API packets dropped at the kernel layer

Impact: NP continuously reports badRidCb errors.

Root Cause: RTCP RID is enabled at the same time as RTP RID is enabled in NP. But when BRM is deactivating the LE2LE_RTCPTERM BRES, it incorrectly sets disableRtcpRid flag to "true" in the RTCP Gen disable command. Even the corresponding RTCP RID was not enabled due to XRM having not resolved the destination MAC address.

Steps to Replicate: 

1. Run negative call tests to reproduce and verify the fix.
2. Make at least 65 of those calls with a remote IP address that is not reachable (i.e., the NRS does resolve the route).
   Note: Considering the NP's badRidCb error reporting threshold is set to 64, ensure to run at least 65 calls in order to see the error in the logs.

The code is modified to allow the BRM code to set disableRtcpRid flag properly in the RTCP Gen disable command.

Workaround: None.

A MAJOR event containing "PORT_RANGE: id xxxxx not found" regularly seen in DBG log

Impact: A MAJOR event containing "PORT_RANGE: id xxxxx not found" is regularly seen in the DBG log.

Root Cause: When a registration expires, there is a 52-second window (based on default retry counts) after the port range control block is deleted and before registration control block is freed. Any SIP subscribe request received within this window is forwarded to egress to use the connection ID of an already-freed port range control block, and results in this error message.

Steps to Replicate:

1. Enable usePortRangeFlag on egress TG.
2. Establish a new registration.
3. Wait for registration to expire (For example, set expiration value to 2100s. You can use the tail command to check the info level DBG log to see the retry message starting).
4. When the retry timer starts, run scripts to send at least 10 subscribe requests.

In addition to above tests, please also verify following:

1. 1. Subscribe refresh is working as expected.
   2. Subscribe request when port range is disabled.

Corrected the logic which only checked registration status when the usePortRangeFlag feature is enabled.

Workaround: None.

ScmProcess may coredump when logging an INFO level message (or Call Trace message) related to secure media IP change after on-hold/off-hold signal sequence

Impact: The ScmProcess may core when logging an INFO level message (or Call Trace message) related to a secure media IP change after a on-hold/off-hold signal sequence.

Root Cause: The IPUtilGetStr function was called to retrieve the previous and current IP for logging, which was used as an argument that got passed into the logging function without any additional parameters. In this scenario, the function uses local memory for storing the value. 

Without specifying a buffer to store the data, the local buffer gets overwritten when called two consecutive times. Consequently, when stringcopy or write is called, the first dataset is gone resulting in a null pointer access that causes a core.

Steps to Replicate: Run at INFO level logging and put secure media leg on-hold and take off-hold to reproduce the core.

The code is modified to remove the IPUtilGetStr from the call to log the DBG message.

Workaround: None.

The SBC application is not coming up

Impact: Synchronization of the SIP REC call data may fail due to deficiencies in the synchronization algorithm.

Root Cause: The algorithm used to the synchronize of SIP REC call data to the standby system suffered from deficiencies, making the algorithm unreliable.

Steps to Replicate:

1. Provision the SBC to perform a SIP REC.
2. Run SIP REC calls.
3. Bring up the SBC's Standby system and/or perform an LSWU upgrade.

The code is modified to correct the algorithm's reliability issues.

Workaround: Disable (inhibit) SIP REC calls before bringing up the Standby system or performing LSWU upgrade.

The SBC 5400 failover occurred on server

Impact: The SCM cores due to memory corruption.

Root Cause: The memory corruption is caused by a bug where the length field of a structure wrapped around the size of the USHORT that it is stored in is causing an incorrect length value.

We use this length to calculate the address where we’re going to copy data into. Therefore, when the length is incorrect the incorrect location data is copied, causing memory corruption.

Steps to Replicate: The steps are not reproducible. 

The code is modified to prevent the length field from wrapping when it becomes very large.

Workaround: None.

SWE Traffic Profile sync problem in HA

Impact: The traffic profile fails to get applied on 1-to-1 SBC 9.2.3R0 Openstack VM registered with V13.02.06R000 EMS when the SBC VM is spawned with the large memory config Profile.

Root Cause: 1-to-1 mode SBC registered with EMS fails to apply the large config profile as part of init operations on an active SBC. This results in the creation of a marker file (skipRebootMarker). The presence of this marker file causes the application to reject the traffic profile activation from CLI/EMS on an active SBC.

Steps to Replicate:

1. Launch 1:1 Openstack SBC with V09.02.03-R0 Build, registered with EMS V13.02.06R000.
2. Log into Active SBC and change sweConfiguration profile to large (both SBCs go for a reboot).
3. Run saveAndactivate from the SBC CLI -- EMS gets synced with the SBC's configuration.
4. Log into Openstack console, and rebuild both Active and Standby SBC instances with same V09.02.03-R0 build image.
5. Log in to the active SBC and check sweConfiguration profile – the profile will be shown as small, rather than large.
6. Try activating the standard_passthrough Profile – none of the SBCs go for reboot for activation of traffic profile.
7. Reboot both active and standby SBC instances manually - only the old active will come up with the activated traffic profile from the previous step.

The code is modified to update the logic for introducing the marker file. The marker file was introduced incorrectly as part of the large config profile activation.

Workaround:

1. Log in to SBCs as root user and remove the marker file from both active and standby instances:
   rm -f /opt/sonus/conf/swe/capacityEstimates/.skipRebootMarker
2. Re-activate the large sweConfigProfile from the active SBCs CLI console by running the following command:
   set system sweConfigProfileSelection name large

Incoming SIP SUBSCRIBE requests from registered endpoints were incorrectly rejected with a SIP 403 Forbidden response 

Impact: The SBC incorrectly rejects incoming SIP SUBSCRIBE requests from registered endpoints with a SIP 403 Forbidden response.

Root Cause: The logic that was added for SBX-119915 to check an endpoint's registration status while processing the incoming SIP SUBSCRIBE request should only apply when the usePortRangeFlag feature is enabled.

Steps to Replicate:

1. Establish a new SIP registration, followed by a new SIP subscription for an event of your choice.
2. Verify that the SBC relays the SIP SUBSCRIBE to the registrar and that the SBC doesn't reject the SIP SUBSCRIBE with a SIP 403 Forbidden response.

Reverted the SBX-119915 code change.

Workaround: None.

Ingress T.38 Fax fails when the Egress is SIP or TDM (SBX-12043) / T.4 data size exceeds expected size based on fax speed (SBX-106015)

Impact:

• SBX-106015 (Sev. 2): The UDP packets for the 14.4 rate are 84 bytes instead of 72 bytes corresponding to a 40 ms payload. This is not a bug, but is considered an unnecessary change that was introduced as a part of T.38 3.35 (This is getting rolled back to the original 72-byte packet size).
• SBX-120493 (Sev. 3): Certain T.38 endpoints send TCF much earlier relative to DCS packets (e.g. fax server, as indicated by 'SIP User-Agent: VFAXSUA/IPFAX-9.0.7806.611'). The current stack handles it by dropping TCF while its still modulating the DCS signal. This results in a shorter modulated TCF, which causes the receiving fax machines to indicate a failure via an FTT message.

Root Cause:

• SBX-106015: The T.38 stack upgrade to 3.35 induced an unnecessary change. While this is not a bug, the best approach is to avoid unnecessary changes.
• SBX-120493: Incorrect DCS message lengths and TCF message timings (TCF messages arrive while the stack is still modulating DCS) are sent from some T.38 endpoints.

Steps to Replicate:

SBX-106015:

1. Establish a. 14.4 rate T.38-G711 fax call so that the SBC stack G.711 leg is Ingress.
   The SBC stack then generates TCF UDPTL packets.
2. Using wireshark, inspect the media packets payload size.
   1. Without the fix, the payload size for media packets is 82 bytes.
   2. With the fix, the payload size is 72 bytes.

SBX-120493:

• If a failure occurs with VFAXSUA/IPFAX type device, and the TCF packets arrive before modulating an earlier DCS, than the generated TCF tends to be shorter than the standard 1.5 seconds.
• With this fix, the TCF duration matches the actual received TCF duration payload.

To overcome when T.38 peers send TCFs earlier than expected, the stack code is modified to delay TCF generation until the DCS is modulated.

Additionally, the T.38 stack code changes are rolled back to produce 72-byte packets for 14.4. TCF.

Workaround:

• SBX-106015: N/A
• SBX-120493: No workaround

SBX-112437

(10.1.0)

The SCM Process and SAM Process core observed for Enhanced Local Redirection

Impact: SCM cores when processing a SIP message that contained 10 Contact Headers and Multiple Identity headers.

Root Cause: The length of the data became too large to be stored in a length field that is a USHORT. The contents of the length field wrapped around the size of the USHORT that it is stored in, causing an incorrect length value.

This length is used to calculate the address where it is going to copy data into. Therefore, when the length is incorrect it copies into an incorrect location causing memory corruption.

Steps to Replicate: Run a test where a SIP message that contained 10 Contact Headers and Multiple Identity headers is sent to the SBC. This message is processed correctly and there was no core.

The code is modified to add defensive code to prevent the length field from wrapping when it becomes very large.

Workaround: None.

SBX-114422

(10.1.4)

SBC 7000 Trunk Group Stats contain Parent CAC trunk group (a.k.a Shared CAC-Limits Pool) stats with strings "key not found in" instead of Zone name and AC name

Impact: SBC 7000 Trunk Group Stats contain Parent CAC trunk group (Shared CAC-Limits Pool) stats with strings "key not found in" instead of Zone name and AC name

Root Cause: The Trunk Group Stats considers sharedCacLimitsPool as a TG and since sharedCacLimitsPool is not linked to address context or zone, those fields are populated with 000, and later while trying to fetch value for the TrunkGroupStatusStats file it is updated as "key not found in."

Steps to Replicate:

1. Set up basic SIP-SIP call.
2. Configure sharedCacLimitsPool.
3. Attach the sharedCacLimitsPool to an Ingress Trunk Group.
   TrunkGroupStatusStats does not contain an Entry for sharedCacLimitsPool.

The code is modified to skip the TrunkGroupStatusStats entry for sharedCacLimitsPool.

Workaround: None.

Unexpected switchover on the SBC SWe pair.

Impact: The SCM process coredumps after processing a Dialog Event.

Root Cause: The cause of the coredump is the code accessing a structure where elements are freed.

As a result, the code de-references pointers in this structure that is set to NULL.

Steps to Replicate: This issue is not reproducible.

The code is modified to free the Dialog structure only after freeing the elements it points to. This fix prevents dereferencing pointers in this structure that are set to NULL. Multiple stack traces show the same routine in the code as being deficient - this is where additional checking logic is added. 

Workaround: There is no workaround.

SBC Switchover processAbnormalTermination with core

Impact: Subscribe relay with multiple crank back failures may cause the SBC to core.

Root Cause: When retransition eventually times out and multiple application crank back failures take place for Subscribe Relay, there may be a core observed due to a stack recursion. This results in a duplicate free of the relay control block.

Steps to Replicate: Not able to reproduce due to specific customer configuration.

The code is modified to process the freeing of the relay control block in a separate thread.

Workaround: Avoid crank back.

SBX-118299

Load Testing - Packet Loss

Impact: SWe_NP was not able to handle traffic burst, and observed packet loss.

Root Cause: Due to high wakeup latencies in SWe_NP caused by scheduling and CPU idle time management, the packets accumulated in the hardware queue are not drained by SWe_NP in time.
The hardware queue is quickly filled up when there is traffic burst, and due the latencies caused by scheduling, SWe_NP could not read the packets in time, and the previous packets were overwritten in the hardware queue.

Steps to Replicate: The reproduction of this test requires simulating the traffic burst.
One way is to use multiple call generators.

The code is modified to change the scheduling policy of SWe_NP to Realtime (SCHED_FIFO) and change the idle behavior of the CPU to poll, instead of halt.

Workaround: Reduce the number of calls and number of call generators.

Running a sysdump on a SBC 5400 triggers packet port down/up condition

Impact: Port link failure observed while running sysdump on the SBC 5400. This could cause an SBC switchover.

Root Cause: Enhancements made to collect more Network Processor (NP) memory area caused interference for the interface driver reading the port link status. This interference caused the driver to falsely report link down. The link status is obtained from the NP processor's (Octeon) register.

Steps to Replicate: Run np_mem_dump.pl while the SBC application is running while the mgt/pkt interfaces are in a link up state.

Sample command line to run 100x:
[root@yf04a np]# x=1; while [ $x -le 100 ]; do echo $x; date; ./np_mem_dump.pl/var/log/sonus/np/npMem0_d$x.log >& npdump.log ; date; $(( x++ )); done

Note: This command creates npMem0_d[1..100].log files and could fill up the disk.

The code is modified to allow the utility function to collect NP memory and NP registers in order to access the Octeon register through the driver instead of directly accessing the register. This avoids concurrent access of the register which caused the false link failure.

Workaround: Do not run sysdump or /opt/sonus/bin/np/np_mem_dump.pl while the SBC is running.

You can obtain an updated /opt/sonus/bin/np/oct-remote-memory and /opt/sonus/bin/np/oct-remote-csr binary to avoid the link bounce while running sysdump.

The SBC performed switchover with processAbnormalTermination error code

Impact: The SCM process cored due to a Healthcheck timeout.

Root Cause: An infinite loop is encountered when mirroring Subscription CBs.

Steps to Replicate: The steps are not reproducible.

The code is modified to prevent the SBC from spinning in this particular loop.

Workaround: None.

After a 491 response due to re-INVITE contention, the SBC continues the call even though the SDP is unmatched.

Impact: After 491 response, the SBC sends re-INVITEs with invalid application media format.

Root Cause: When the application processes the queued request internally, it loses the other leg application media format that it used for the remote to send out.

Steps to Replicate:

1. Enable e2e re-INVITE.
2. Run multimedia SDP applications.
   After the call connects with both media, both ends try to re-INVITE with one media active that will trigger the SBC to send 491 and re-INVITE.

The code is modified to properly store the other leg application media format in SIPS, so that it can send out correctly when it processes the queued message.

Workaround: None.

The SCM Process cores and causes a switchover.

Impact: The SCM Process cores when attempting to allocate memory.

Root Cause: The SCM Process cores when attempting to allocate memory because the memory management code has detected memory corruption.
The memory corruption occurred because the code was written into a memory block after it had been freed. This appears to have occurred when the SRS gets blacklisted during setup and the call needs to route advance to the next SRS.

Steps to Replicate: The steps are not reproducible.

The code is modified to prevent writing into this memory block after it is freed.

Workaround: None.

The SBC Core has a switchover twice during the day.

Impact: SCM cores in SIP code when using SIP Refer Relay.

Root Cause: The core is due to an attempt to dereference a NULL pointer while processing a message related to SIP Refer Relay.

Steps to Replicate: The root cause was found by code inspection.
The exact call flow that will causes the pointer to be NULL when entering the affected code is unknown.
(With this fix, the code will not core in the future if the pointer is NULL.)

The code is modified to make sure the pointer is valid before attempting to dereference it.

Workaround: Disable the SIP Refer Relay.

SIPSG code cores due to accessing pktCcb pointer that has been set to NULL

Impact: SIPSG code cores due to accessing the pktCcb pointer that has been set to NULL.
This issue can show up with a number of different stack traces (depending on the call flow).

Root Cause: The fix in the past included a change that sets fsmCcb->pktCcb to NULL after clearing out the contents of the pktCcb.

The reason for setting fsmCcb->pktCcb to NULL after clearing out the contents of the pktCcb was to prevent executing the code in SgOaClearPktCcb() more than once.

The previous fix was unnecessary because SgOaClearPktCcb() already checks every field for NULL before attempting to free it.

This change causes new cores because there is code that is accessing pktCcb pointer after it was set to NULL (in the past it was safe to assume that pktCcb pointer would not be NULL).

This bug was introduced in the 9.2.4R3 release.

Steps to Replicate: The steps are not reproducible

The code is modified to remove the change that sets fsmCcb->pktCcb to NULL.

Workaround: None.

Congestion Memory issue.

Impact: The SCM process leaks memory when "sendSBCSupportedCodecsForLateMediaReInvite" is enabled in the IP Signaling Profile.

Root Cause: The SCM process may not free internally allocated structure when "sendSBCSupportedCodecsForLateMediaReInvite" is enabled in IP Signaling Profile.

Steps to Replicate: Run load with “sendSBCSupportedCodecsForLateMediaReInvite” enabled in the IP Signaling Profile and confirm the leak.

NOTE: The call flow must result with sending a "NRMA Modify" internally.

The code is modified to ensure that these structures are freed when "sendSBCSupportedCodecsForLateMediaReInvite" is enabled in IP Signaling Profile.

Workaround: Disabling "sendSBCSupportedCodecsForLateMediaReInvite" may prevent this issue.

However, there may be other code paths or scenarios which may trigger this leak.

A switchover occurs on the SBC due to a deadlock issue.

Impact: A switchover occurs on the SBC due to a deadlock issue.
The CPX process deadlocks while calling CpxTrunkGroupMediaIpSecondaryInterfaceGroupNameValidate to validate the SecondaryInterfaceGroupName while the customer was making lots of calls to get the authPassword, for example:

show table addressContext <context> zone <zone> sipTrunkGroup <trunkgroup> signaling authentication authPassword

Root Cause: When the the authPassword is being fetched multiple times a second, that call was occuring at the exact same time as the CpxTrunkGroupMediaIpSecondaryInterfaceGroupNameValidate was called. Both routines use the same global maapiSocket to do maapi calls to read the confd database. Since the same global socket was being used, the maapi_exists call is hung.

Steps to Replicate: In one CLI session, make a lot of calls to get the authPassword:
show table addressContext <context> zone <zone> sipTrunkGroup <trunkgroup> signaling authentication authPassword

In another CLI session, do a:
set addressContext <context> zone <zone> sipTrunkGroup <trunk group> media mediaIpSecondaryInterfaceGroupName <groupname>

Observe that the CPX does not crash.

The code is modified so that the CpxTrunkGroupPassGetElem function performs a 'cdb get' action to get the authPassword, so there is no contention for the global maapiSocket.

Workaround: Do not get the authPassword at the same time as setting the mediaIpSecondaryInterfaceGroupName.

After an SBC 9.2.3 SW upgrade, the ACLs are blocking traffic when enabled.

Impact: The IPACL rule is incorrectly installed in the NP.

Root Cause: You can configure the IPACL rules with or without an ipInterface. If an IPACL rule is configured with an ipInterface, it is not installed in the NP until the associated ipInterface is created and enabled in the NP. The IPACL task tries to re-add those rules that are waiting on the ipInterface for it to receive a 'port up' notification from the NRS.

When mixing a set of IPACL rules, some rules that are not defined with the ipInterface are installed in the NP during the startup of the applications, while other rules that are defined with the ipInterface may get delayed while waiting for the associated ipInterfaces to become ready. All of the IPACL rules are ordered by precedence in the CPS context and will be programed in the NP in the same order.

When the IPACL task tries to re-add the rules that are waiting on ipInterface, the order of the rules is changed. The new rules are inserted at the proper location in the CPS context, based on their precedence. But there is no corresponding shuffling at the NP layer which causes some newly-added rules to randomly overwrite some existing rules.

Steps to Replicate:

1. Configure the following three IPACL rules:
   rule 1: Define the LIF and the ipInterfaceGroup that contains this LIF, and assign the smallest precedence value among three rules.
   rule 2: Use the same ipInterfaceGroup as rule1, protocol = ICMP, assign precedence value > rule1, do not define as ipInterface.
   rule 3: Same as rule 2, except assign the precedence value > rule2.
2. Take the LIF OOS and disable.
3. Restart the application.
4. Enable and in-service LIF.
5. Ping the next hop address through pkt port of the LIF.
6. Check the stats using the following command:
   "show table addressContext ADDR_CONTEXT_1 ipAccessControlList ipAclRulesByPrecedence"

This should result in rule 3 showing match counts instead of rule 2.

The code is modified to add all the rules in the NP, including the ones already added on the IPACL retry.

Workaround: None.

The SBC application restarts.

Impact: A SAM process encounters a core due to a Healthcheck timeout when the SIPCM attempts to open a TLS connection. The security certificate named in the tlsProfile is expired.

Root Cause: Two threads are hung while attempting to write a security certificate update () to the DB using the same socket. The purpose of the update is to change the state of the certificate to EXPIRED.

Steps to Replicate: 

1. Run a TLS load that will attempt to use an expired PKI security certificate (tlsProfile includes the name of the security certificate).
2. Look for the following messages in the DBG logs:

100000B.DBG:076 05262022 000010.545176:1.02.00.18170.MAJOR .SEC: *Certificate expired

The code is modified to add a lock preventing two threads from writing to the DB using the same socket.

Workaround: Renew PKI Security Certificates before they expire.

SBC Ethernet Port Packet Port Statistics have peak value equal zero.

Impact: SBC Ethernet Port Packet Port Statistics have incorrect peak value equal to zero.

Root Cause: The SBC has reset the bandwidth peak value after statistics collection.

Steps to Replicate: Execute a call load for hours and collect ethernet port packet stat.

The code is modified allowing the SBC to reset the bandwidth counter before it collects the real data.

Workaround: None.

Node B unexpectedly restarts when Node A is in SyncInProgress for a long time.

Impact: The Call/Registration Data on the SBC shows syncInProgress for an extended period of time.

Root Cause: A redundancy message buffer is not large enough to hold the data and causes the sync to be incomplete. This results in the Call/Registration Data to remain in a syncInProgress state.

Steps to Replicate: Run multiple Registration/Calls on the SBC.

The code is modified to address the issue. 

Workaround: None

Application restarted due to processAbnormalTermination

Impact: SSREq process coredumped while receiving message with length =0.

Root Cause: When the message length is =0, the third party software, Xalan, crashes when allocating memory.

Steps to Replicate: Perform SSREq request to ERE when configured for external PSX.

The code is modified to add a check to not process messages with length=0.

Workaround: Stop the SSREq process.

The SBC resets SRTP encryption sequence number after a re-INVITE; the anti-replay protection at the far end (on another SBC) discards the incoming SRTP packets until the sequence number approaches the highest recorded sequence number.

Impact: Intermittent one-way audio with a duration of more than 10 seconds may occur during call transfers when the remote connection media IP address changed on the secure RTP (SRTP) call leg. The RTP sequence number (SSN) of the outgoing SRTP stream encrypted by the SBC may roll back to a lower value that causes the SRTP packets being discarded by SRTP anti-replay protection at the receiver's end. Once the SSN of the SRTP stream reached the highest recorded value, the audio was back and both parties could hear each other.

Root Cause: After the call transfer, the remote connection media IP address on the SRTP call leg was modified without deactivating the media resource chain. This caused the delay in applying the security configuration to the media resource chain, which in turn caused the first one of several SRTP packets being sent with a high SSN number and the roll back of the SSN to a lower value. The remote end discarded the SRTP packets with a lower SSN than the highest recorded SSN.

Steps to Replicate: 

1. Make a SRTP-to-RTP call between A and B through the SBC under test.
2. A puts B on hold and calls C through a different SBC/device.
3. A bridges B and C together and uses a late-media re-INVITE towards B to modify the original connection media IP address with the C's media IP address.
4. The SBC sends a re-INVITE with a SDP offer to B.
5. Once the SBC receives a 200 OK with a SDP answer from B, it starts sending the SRTP packets to C's connection media IP address.
6. The first SRTP packet sent to C has a SSN of around 1000 and the second SRTP packet has a SSN of less than 256. C discards all received SRTP packets until the SSN reaches the highest recorded SSN, which is 1000.

The code is modified so in a modify offer or answer for a SRTP call, if the remote connection media IP address is modified on the SRTP call leg, the new IP and security configuration are applied at the same time, without a delay.

Workaround: None.

SBC delay in processing Late Media re-INVITE

Impact: The SBC is not able to relay/responds to the late media re-INVITE after call is connected.

Root Cause: There is a race condition where ingress connects first before egress. The SBC received reInvite and forwards to egress. Egress internally responds 491 because egress is not in the connected state yet. However, internally the CC subsystem is ignoring to relay the 491 and responds due to the wrong state.

Steps to Replicate: Configure the passthru latemedia:

1. Late media call
2. Ingress sends latemedia re-INVITE immediately after sending ACK for the first Invite.

The code is modified to correct the CC logic to relay the 491 responds to ingress.

Workaround: Disable passthru latemedia.

Transcoding is failing for T.38 involving SRTP

Impact: The SBC is not sending SRTP attributes in an outgoing offer for the G711 fax call.

Root Cause: This issue is specific to a call flow where the initial audio call is a pass-through with non-G711 codec and later transitions to fax using G711. In this call flow, the SBC fails to offer SRTP crypto attributes during the fax call.

Steps to Replicate: Configuration:

Ingress Route PSP -
Fax Tone Treatment: FAX_TONE_TREATMENT_FALLBACK_G711

Egress Route PSP -
Fax Tone Treatment: FAX_TONE_TREATMENT_IGNORE_DETECT_ALLOW_FAX_RELAY

PSP -> Enable flag "Apply Fax Tone Treatment"

1. Setup G.722 pass-through call with SRTP on ingress leg.
2. Once audio call is stable, send T.38 re-invite from UAS.
3. The SBC sends G711 fax re-Invite towards ingress leg.
   
   

Without fix: The re-Invite at step 3 is sent with out SRTP.

With fix: The re-Invite at step 3 is sent with SRTP.

The code is modified to offer crypto attributes even when the initial audio call is using a non-G711 codec.

Workaround: None.

SIP to SIP call had no response to PRACK.

Impact: The SBC is unable to respond to PRACK with new SDP offer while the call is in terminating state.

Root Cause: Call is in terminating state.

Steps to Replicate: The Ingress configuration disconnected the treatment.

1. A (PRACK support) call B, B answer failure response (ex: 486)
2. SBC sends 183 to play disconnected treatment.
3. Peer sends PRACK with SDP new offer.

The code is modified so the SBC rejects a 500 error response to a PRACK request with an SDP offer.

Workaround: Disable PRACK or disable disconnected treatment.

Video call disconnects abruptly due to ACK not getting propagated

Impact: The SBC is unable to answer ACK for re-INVITE on ingress leg.

Root Cause: The Egress leg is resetting the end-to-end ACK feature after receiving 200 OK offer for latemedia call. As a result, this causes the ACK to not send on Ingress for re-INVITE.

Steps to Replicate:

1. Configure the latemedia passthrough, enable e2e ACK, and e2e re-INVITE
2. Run a late media call
3. Egress sends a re-INVITE to Ingress
4. Ingress answer 200 OK
5. Egress answer ACK

The code is modified to turn on the e2e ACK feature back after the ACK send out for latemedia call.

Workaround: Disable e2e re-INVITE or e2e ACK.

SAM Process coredumps are causing a switchover if FAC enabled and call Id is longer than 24 characters.

Impact: While reporting statistics for a Fault Avalanche Control (FAC), the SAM process may core if reporting on a call with a callId larger than 24 characters.

Root Cause: While reporting statistics for FAC, an incorrect "maximum size" is used when copying a callId - resulting in the code writing past the end of an allocated buffer and corrupting memory.

Steps to Replicate: Run some calls with FAC while collecting FAC statistics.

The callids for these calls should be longer than 24 characters.

The code is modified to use the correct "maximum size" when copying the callId.

Workaround: Disable FAC.

The FAC feature blocks SIP messages of certain key elements that caused multiple coredumps in the past. Disabling FAC will block them from getting processed again

An SCM Process core occurred on both servers.

Impact: The SIPCM process cores.

Root Cause: The SIPCM process cores because it was accessing a Packet CCB that had already been freed.

Steps to Replicate: The steps are not reproducible.

The code is modified to add defensive code to prevent SIPSG from accessing a Packet CCB that is already freed.

Workaround: None.

The SBC cores along with disk errors seen (2ndcore)

Impact: The SCM cores while attempting to update SIP protocol specific fields in the accounting record.

Root Cause: The code is using the wrong function to get a pointer to the URL header structure.

Therefore, the SBC is using an invalid pointer and operating on invalid data.. This resulted in an attempt to dereference a NULL pointer, which caused a Segmentation Fault.

This code path is executed when writing SIP protocol specific fields into accounting record.

Steps to Replicate: This issue is not reproducible. The code has been the same since prior to the 7.2 release and this issue hasn't been seen before. Low risk of reoccurrence.

The code is modified to use the correct function to get the pointer to the header.

Workaround: Disable CDRs.

sanitizer.CE_2N_Comp_CpxAppProc & CE_2N_Comp_DiamProcess

Impact: A small amount of memory leaks while configuring diameter peer IP addresses

Root Cause: The configuration validation routines were allocating memory while cross-checking the new CDB configuration and this memory was not freed at the end of the validation.

Steps to Replicate: Create diameter peer IPs and check for no memory leaks in an ASAN enabled build.

The code is modified to update the validation routines to make sure the memory is freed so there is no leak.

Workaround: None.

LeakSanitizer in CE_2N_Comp_CpxAppProc, CE_2N_Comp_ScmProcess0,1,2,3

Impact: While sending a SIP SUBSCRIBE message, the code was leaking memory if there were PATH headers in the associated subscribed registration data.

Root Cause: The code was creating copies of the PATH header information to map into the P-Asserted-ID header and not freeing up all the associated information once the subscription processing completed.

Steps to Replicate: Run various SUBSCIRBE for REGISTER event test cases and check there are no memory leaks.

The code is modified to make sure all P-Asserted-ID header memory is correctly freed up.

Workaround: None.

SBX-115152

(8.2.6)

Unknown header is not relayed with re-INVITE call flow in PSX.

Impact: Unknown Header Transparency for BYE message do not work in call scenario where SBC receives a re-INVITE from Egress peer and BYE is received from Ingress Peer. Issue is observed only when Re-INVITE is auto-answered by SBC

Root Cause: SIP request data for Re-INVITE is not cleared from SIP call control block after Re-INVITE offer-answer completion. This results in duplicate entries for SIP request data in the SIP call control block when the SBC receives the BYE message. During handling of the BYE message at Egress leg, request data pointer is retrieved from call control block to set transparency mask for unknown header, request data pointer of Re-INVITE is picked always instead of BYE message. This results in unknown header transparency issues for BYE.

Steps to Replicate: 

1. Make SIP-SIP call setup with G711 U law.
2. Enable "Minimize Relaying Of Media Changes From Other Call Leg" flag in Ingress IPSP.
3. Enable "Unknown Header" transparency in IPSPs.
4. After session establishment, send Re-INVITE from Egress peer with G711U law for call hold ( a=sendonly).
5. Send BYE from Ingress peer with Unknown Header.
6. Verify if unknown header in BYE message is sent transparently to other leg.

The code is modified so that SIP request data is cleared properly for Re-INVITE.

Workaround: None.

SBX-120128

(9.2.4)

The SBC 5400 switchovers occur after an upgrade to v09.02.04R003.

Impact: The SCM process may core while attempting to set the original peer SDP address when either NAPT for Media is enabled and the SDP is received, or if ICE is enabled and ICE is in the SDP.

Root Cause: In the above two scenarios, the software did not protect against using a NULL packet SG control block address.

Steps to Replicate: The steps are not reproducible.

The code is modified to prevent the NULL packet SG control block address from causing the SCM process to core.

Workaround: None.

SBX-115304

(8.2.6)

The SBC replies with a 481 message for INVITE with occasional replaces.

Impact: The SBC replies a 481 message for INVITE with occasional replaces.

Root Cause: In early dialogue, the state to-tag is not updated in the Hashtable but the from-tag is updated. This issue is why when we tried to look up using only the call ID. As a result, it fails to fetch original call as the original INVITE and the INVITE with replaces had the same call ID.

Steps to Replicate: 

1. Prepare a SIP-to-SIP call setup with INVITE with replaces from the client side having the same call ID for original INVITE and INVITE with replaces.
2. Check if the INVITE with replaces is replacing the original call and if the lookup is successful or not.

The code is modified to look up both the call ID and from-tag for the ingress leg.

Workaround: None.

SBX-118049

(10.1.2)

The SRTP decryption and encryption fail after an upgrade to 9.2.4R0/10.1.0R2.

Impact: The SBC fails to decrypt the incoming SRTP packets, causing a static noise instead of audio on the SRTP calls. 

Root Cause: Since 9.2.0x, one flag is used in the XRM. XRM_SRTP_SESS_UNENCRYPTED is used for both the unencrypted SRTP and unencrypted SRTCP. When this flag is set, the XRM sends cipherType = NP_CIPHER_TYPE_sRTP_NULL to the NP. The SBC does not to decrypt the incoming SRTP packets.

Steps to Replicate: 

1. Enable the UNENCRYPTED_SRTCP in the cryptoSuiteProfile.
2. Enable the INFO level DBG logging and make calls.
3. Check the DBG log for NpMediaRidEnable and NpMediaFlowEnable messages
4. Ensure that:
   • RTCP = cipher/size 0x8000000/0x40000
   • RTP = cipher/size 0x6000000/0x40000

A new flag is added, for unencrypted SRTCP: XRM_SRTCP_SESS_UNENCRYPTED

The modified NRMA and XRM adopt the new flag for SRTCP.

Workaround: Disable the UNENCRYPTED_SRTCP in the cryptoSuiteProfile

SBX-115231

(11.0.0)

Possible stack overflow vulnerability in the RTCP processing.

Impact: Potential stack overflow vulnerability was identified through the internal code review in the RTCP generation subsystem of SWe NP module.

Root Cause: The root cause of the problem was identified to the use of variables of an incorrect storage size.

Steps to Replicate: The steps cannot be reproduced. 

The code is modified to address the vulnerabilities.

Workaround: No workaround.

SBX-114456

(8.2.6)

The AMRWB is transcoded although negotiated end-to-end.

Impact: The Transcode option for "different2833PayloadType" feature considers only the 8,000 DTMF payload and the 16,000 DTMF payload is not compared with the opposite leg Peer PSP DTMF, unlike the 8,000 DTMF in earlier releases. This leads to unnecessary transcoding when the same 16,000 DTMF is present in both Ingress Offer and Egress Answer.

Root Cause: The SBC does not compare the 16,000 DTMF payload type of Ingress and Egress Leg when "different2833PayloadType:" is enabled, unlike 8,000 DTMF. The SBC should only transcode the call only when the 16,000 DTMF payload is different for both Ingress and Egress; otherwise, the SBC should treat the call for pass-through (PT).

Steps to Replicate: 

TC1:

1. Set preferred DTMF to 101 and enable different2833PayloadType flag.
2. From Ingress Peer send PCMU and AMRWB with 8,000 PT 101 and dtmf PT 16,000 102.
3. Egress Peer send AMR-WB 102 in answer.
4. The call should be pass-through with DTMF 102.

TC2:

1. Set preferred DTMF to 101 and enable different2833PayloadType flag.
2. From Ingress Peer send PCMU and AMRWB with DTMF 8,000 PT- 104 and DTMF16K PT - 102.
3. Egress Peer send AMR-WB 102 in answer.
4. The call should be pass-through with DTMF 102.

TC3:

1. Set preferred DTMF to 101 and enable different2833PayloadType flag.
2. From Ingress Peer send PCMU and AMRWB with DTMF 8,000 PT- 104 and DTMF16K PT - 105.
3. Egress Peer send AMR-WB 102 in answer.
4. The call should be transcoded for difference in DTMF - 102 in Egress and 105 in Ingress.

The code is modified to check both the 8,000 and 16,000 DTMF for a difference in payload type with the corresponding answered DTMF payload types, sent by Egress Peer in Offer Answer function to determine whether the call should be transcode or pas-through. If the DTMF payload types are same, the call is passed through and transcoded if the DTMF payload types are different. These checks are done when "different2833PayloadType" feature flag is enabled.

Workaround: None.

SBX-115616

(10.1.1)

Routing label settings could not be changed in the EMA if created by the CLI.

Impact: Unable to change Routing label settings from the EMA UI if they were created using special characters  in the CLI.

Root Cause: CLI was allowing _ + - . : @ characters in the Routing Label name. The EMA, however, was only allowing  _ character in Routing Label name. As a result, attempts to update the Routing Label from the EMA failed.

Steps to Replicate: 

1. Log in to CLI.
2. Created Routing Label with following special characters (+:.@-) from CLI.
3. Log in to the EMA.
4. Navigate to Routing and select the Routing Label and edit the same.
5. Edit is successful.

The code is modified to allow special characters + - . : @ in the EMA.

Workaround: No workaround.

SBX-117637

(11.1.0)

The SCM Process generates a core and a switchover occurs due to null pointer in a forked call.

Impact: The SCM is coring when trying to free memory that has already been freed.

Root Cause: The SCM process cores when trying to free memory because of a non-reproducible edge case scenario which causes the memory to be freed before returning from a subroutine. The calling function doesn't handle this correctly.

NOTE: This scenario only happens if a downstreamForkingSupport is enabled on the SIP Trunk Group.

Steps to Replicate: The steps cannot be reproduced. 

The code is modified to prevent the scenario.

Workaround:  None.

SBX-115262

(8.2.6)

Unexpected UPDATE from the SBC after UPDATE with media inactive.

Impact: Delayed RBT with the monitor profile in early dialog with DylRBT, there is an update from the caller to set the media to inactive. That triggers an UPDATE towards the called with media inactive. Just after that, the SBC sends a new UPDATE with the media direction "sendrcv" to the caller.

Root Cause: Since in this case tone is playing and dlrbt is enabled, Update from ingress with DPM inactive is processed by the SBC and sent to egress. Egress responds with 200 OK with Media as inactive which is passed to ingress.

The SIPSG sends notification to CC to STOP tone. Tone stops and cut-thru is set in CC. CC sends activate to NRMA and NRMA swaps tone context with cut-through context that has a DPM of sendrecv. NRMA sends update notify ingress with DPM of sendrecv, which is sent to ingress peer as an Update.

Steps to Replicate: 

1. Run the Customer configuration.
2. Run the Customer Scenario DelayedRBT enable with Tone continuation and Monitoring profile is enable.
3. Run an additional update to stop after an UPDATE with media = inactive.

The code is modified to address the issue. 

Workaround: No workaround.

SBX-114764

(8.2.6)

The G711 codec is used incorrectly during a G729 UPDATE.

Impact: The SBC plays the incorrect media towards Ingress End Point after a call is established.

Root Cause: Issue 1: Port is not updated after an UPDATE during tone play.

After the SBC receives an UPDATE from Ingress End Point during tone play, the SBC updates the codecs and port information. But after a 200 OK for UPDATE is sent out, the SBC overwrites the new information with old information.

Issue 2: The wrong codec is used after the call is established.

After an UPDATE is received during a tone play, the SBC tends to suppress sending UPDATE/re-INVITE towards ingress during cut-thru (after tone play) by re-using the last tone codec on ingress leg.

However, it does not preserve/re-use the transcoding resources which results in wrong media being played towards Ingress EP.

Steps to Replicate: 

1. Configure the SBC with Basic call configuration.
2. Attach TONE_GENERATION_CRITERIA to Tone and Announcement Profile.
3. Configure earlyMedia in Egress TG.

Procedure:

1. The SBC receives INVITE with 0,8,18 from Ingress EP.
2. The SBC receives 180 Ringing with 0 from Egress EP.
3. The SBC receives UPDATE from Ingress EP with 18 and with new media port.
4. The SBC receives 200 OK for INVITE without SDP from egress EP.

Expected Results:

1. The SBC will forward the INVITE to Egress EP.
2. Once the SBC receives 180 with SDP, it will forward to Ingress EP.
   The SBC starts monitoring RTP from Egress. Once timeout happens, the SBC will start generating tone towards Ingress EP.
3. Once an UPDATE is received, the SBC will change the codec to 18 and port to new media port.
4. The SBC receives 200 OK for an INVITE.
   The SBC will use the same codec combination used during tone play.
   A call gets established successfully as G729-PCMU.

The code is modified for the following issues: 

Issue 1: The SBC now changes the port towards Ingress EP.

Issue 2: The correct media is played at the tone play Ingress EP, based on whether or not the call needs transcoding resources..

Workaround: None.

SBX-118191

(11.0.0)

An upgrade is successful but revert operation failed - V10.1.0R2 to V11.0.

Impact: The SBC application does not come up after revert on cloud 1:1 mode.

Root Cause: The SBC application gets stuck due to Openclovis model update cache on old active disk.

Steps to Replicate: 

1. Perform an upgrade on an SBC HA 1:1 on public cloud (such as AWS) from a pre-9.2 to a 9.2 release.
2. Revert the SBC HA 1:1 to a pre-9.2 release.

The code is modified to indicate this model update cache and perform a model update on standby after revert to remove any component that is not needed.

Workaround: 

1. Bring up SBC HA application with a pre-9.2 release on public cloud, such as AWS.
2. Upgrade both active and standby SBC to this release using the recommended upgrade method.
3. Revert the SBC application to pre 9.2 release.
4. After revert operation, check to see if application is coming up fine on both active or standby.
5. If the application is stuck and not able to get the assigned role, stop SBC application on both active and standby nodes using ‘sbxstop’ command.
6. Remove the following file/directories from both active and standby SBC if existing:
   1. rm -rf /home/cnxipmadmin/peerDynamicHANewComps
   2. rm -rf /opt/sonus/sbx/openclovis/var
7. Start the SBC application on both active and standby nodes using ‘sbxstart’ command.
8. Check to ensure that application is coming up fine on both nodes.

SBX-111832

(10.1.0)

PRS Process and ENM Process coredumps observed in SLB during REGISTER Performance run

Impact: The PRS and CHM processes were allowing switchover even when the sync was in progress, causing failures in restoring context in XRM.

Root Cause: The sync completion check was incorrect and incomplete, thereby allowing a switchover to occur even when the sync is in progress.

Steps to Replicate: Trigger the switchover while the sync is still in progress to check if standby either takes over, or if it restarts to prevent the switchover.

The code is modified to add a thorough sync completion check to the PRS and CHM processes to prevent a switchover when a sync is in progress.

Workaround: None.

SBX-118118

(9.2.4)

Both of the SBC units went down after an unregister from the EMS.

Impact: Both of the SBC units went down after an unregister from the EMS

Root Cause: The code in the ENM process notifies the deletion of the SnmpAgent snmpTargetMib snmpTargetAddrTable snmpTargetAddrEntry. The ENM process code spawns a thread to delete the corresponding trap from the oam snmp trapTarget table that shares a CDB socket with the SonusSnmpTrapTarget::ProcSubDelete.

Steps to Replicate: 

1. Register an SBC instance with an EMS.
2. Unregister the same SBC instance with the EMS.
3. Observe that the ENM process does not crash.

The SonusSnmpTrapTarget::DeleteTrapTargetThread routine is modified to use its own CDB socket to prevent contention from two threads for the same socket.

Workaround: None

SBX-114778

(10.1.1)

The STOP RECORD is missing from CDR VIEWER after a switchover.

Impact: STOP RECORD IS MISSING from CDR VIEWER after a switchover.

Root Cause: With the current Logstash configuration, the logstash reads the TRC and ACT files only from the time when the logstash was started, which means older data is not read by logstash. When a switchover occurs, by the time logstash is started all the calls would have completed thereby updates to TRC and ACT would have stopped.

Logstash does not read the TRC and ACT files until they are updated again. When the updates start, logstash does read from the point where the new data is written and not the old data. This is the reason why stop record of the calls are not shown after the switchover

Steps to Replicate: 

1. Log in to the EMA of an active SBC and Enable CDR Viewer.
2. Run a call with a duration (about 50 seconds) such that it gets stopped before EMA is started in the standby SBC after the switchover.
3. Initiate a switchover.
4. Log in to EMA of a standby SBC (which is the active now).
5. Navigate to CDR viewer screen.

The STOP record should display.

The code is modified to read the older data and not just the new data.

Workaround: None.

SBX-110857

(8.2.6)

The RTP stream with unknown port (5004) with Delayed RBT after 180.

Impact: RTP packets being sent to port 5004 after 180.

Root Cause: The RTP packet was the silence packet generated from DSP. Before RTP monitoring was enabled, the call had already been cut through. When RTP monitoring was enabled, NRMA sent a flow change to XRM to change the remote port to 5004. As a result, the DSP-generated silence packet got sent out to remote peer on port 5004.

Steps to Replicate: 

1. Enabled INFO debug logging.
2. Establish a simple transcode call with RTP monitoring enabled on egress side.
3. Check the DBG log for the port number in message "NpMediaFlowModify: Src RTP port modified port 5004 enSrcVal 0x1"

The code is modified so the XRM ignores the remote RTP port change. So the RTP packets are sent to resolved remote IP and port.

Workaround: None.

SBX-105749

(8.2.6)

The RTP monitoring failed when the iteration is set to 1 or 2.

Impact: The RTP Monitoring is not working as expected when the iteration is set to 1 or 2

Root Cause: When the monitoring period is over, it sends FIRST expiry and FINAL expiry together. As a result, this is not being handled properly leading to non-playing of LRBT.

Steps to Replicate: Configure monitoring cycle to 1 and make calls. Ensure that LRBT is heard when monitoring cycle gets over.

The code is modified to play the tone.

Workaround: Use n=3 or more.

SBX-120264

(9.2.4)

The SCM Process continually cores on two SBC 7000s - possibly PCV IOI related.

Impact: The SCM Process coredumped due to NULL pointer access for a (GSX) ISUP-GW-GW-SIP call after enabling Generate or Replace PCV feature.

Root Cause: The code was trying to read a CPC parameter from the ingress side of the call and the parameter is NULL when the ingress side is ISUP.

Steps to Replicate: 

1. Set up a GW-GW call from GSX using ISUP as ingress
2. Enable Signaling > Generate or Replace PCV

Expected result: The SCM Process cores.

The code is modified to ensure the CPC parameter is not NULL before accessing it.

Workaround: Disable Signaling > Generate or Replace PCV and apply the fix.

SBX-118822

(9.2.4)

The HA SBC fails to start on version 09.02.04R001 when the Direct IO Passthrough is used.

Impact: The SBC application fails to start on a 44 vCPU instance.

Root Cause: The maximum number of vCPUs supported is set to 40 in the SM module. This leads to corruption and a subsequent core-dump on instances with a size greater than 40 vCPUs.

Steps to Replicate: 

1. Install the SBC application on an instance with a vCPU size greater than 40.
2. Make sure the application starts and there are no core dumps observed.

The code is modified and now set to be in sync with other modules.

Workaround: None

SBX-118367

(9.2.4)

Missing CDR sub fields from Column: Ingress Protocol Variant Specific Data.

Impact: Missing CDR sub fields from Column: Ingress Protocol Variant Specific Data.

Root Cause: The CDR fields were not updated in SipSgFillProtocolSpecData function from where CDR was getting populated.

Steps to Replicate: 

1. Replicate the issue by running a basic SIP-to-SIP call and removing any mandatory header field.
2. Check if all of the 86 sub fields are getting populated in CDR of 45th field (Ingress Protocol Variant Spec Data).

Updated SipSgFillProtocolSpecData function with the CDR field that were missing to address the issue.

Workaround: None.

SBX-119525

(11.1.0)

Observed the SCM Process core while running callflow sanity on the SBC 7000

Impact: Application gets crashed while running call flow sanity on the SBC 7000.

Root Cause: During Call Progress when a timeout is hit and CANCEL is triggered towards endpoint then due to invalid/null pointer access core dump is observed.

Steps to Replicate: Run a call flow sanity on an SBC 7000.

The code is modified to ensure the pointer is validated for NULL before accessing it.

Workaround: None.

SBX-114415

(8.2.6)

The SBC 7000 standby CE stopped functioning with file corruption and "read only" mode.

Impact: When the file system turned to Read Only mode the server did not go for reboot.

Root Cause: The SBC became stuck during the read only file system checking but the root cause cannot be determined.

Steps to Replicate: The steps cannot be reproduced.

The code is modified to use different commands to check for read only status to try and ensure the SBC automatically reboots to recover.

Workaround: An operator needs to manually reboot the SBC in order to recover it.

SBX-113060

(10.1.2)

The SBC needs to open both protected TCP and UDP port in AKA mode.

Impact: The SBC drops messages coming on the TCP when some phone does initial registration over UDP but later sends message over TCP in IMS AKA scenario.

Root Cause: When initial REGISTER comes on UDP in IMS AKA scenario, the SBC does not open protected ports on TCP. This will cause problem with some phones, which do initial registration over UDP, but later may send some messages over TCP during call.

Steps to Replicate: 

1. Run an IMS AKA registration over UDP.
2. Check netstat on the SBC to see if the negotiated protected ports are opened on both UDP and TCP.

The code is modified so that the SBC opens protected ports on both TCP and UDP when the initial registration is over UDP.

Workaround: None.

SBX-118988

(11.0.0)

The sysDump saved with an owner as root even though sbcDiagnostic 2 is run with linuxadmin as user.

Impact: In the Cloud SBC when sysdumps are generated using sbcDiagnostic command as linuxadmin user, linuxadmin is unable to copy the sysdumps due to restricted permissions.

Root Cause: sbcDiagnostic was generating sysdumps with root ownership, which did not allow linuxadmin to copy them.

Steps to Replicate: 

1. Run sbcDiagnostic 2 command as linuxadmin user in the cloud SBC.
2. Try copying generated sysdumps from /opt/sonus/external/ location.

With a fix, linuxadmin should be able to copy sysdumps.

The code is modified to allow copying.

Workaround: None.

SBX-118635

(11.0.0)

Observed the IpSockAddr (0.0.0.0) and Port (0) for Egress leg in GW1 and Ingress leg in GW2 for GW-GW HPC call scenario.

Impact: The IpSockAddr and Port fields are 0 in the global callDetailStatus status command in a GW-GW HPC call scenario.

Root Cause: In 9.2.4R001, the code is added to not send CPC_OP structures unknown to the GW-GW encoder layer over the GW-GW link to reduce the size of the GW-GW message sent to older SBCs and the GSX. However, it was found that at least one of these unknown CPC_OP Structures, CPC_OP_SIP_ZONE_AC_NAME_STR was used to transfer information used in the global callDetailStatus status command.

Steps to Replicate: 

1. Set up two SBCs that communicate over the GW-GW.
2. Place a GW-GW HPC call scenario call.
3. Observe that the IpSockAddr and Port fields are not 0 after running the global callDetailStatus status command.

The code is modified to send the CPC_OP structures unknown to the GW-GW encoder layer over the GW-GW link.

Workaround: None.

SBX-116340

(9.2.3)

Customer reported DNS failures on 3 different occasions that appears to be related to "time to live" and negative cache functionality.

Impact: DNS records (requests) can become stuck in RESOLVING state within the Agent, which causes the FQDN to become unreachable forever.

Root Cause: A DNS request (or it's response), sent between the Agent (e.g., ScmProcess) and the Client (DNS Process), becomes lost. This causes the DNS record to become stuck in RESOLVING state, and the FQDN to become unreachable.

Steps to Replicate: The steps cannot be reproduced.

The code is modified to:

• Add a default value of 60 seconds for the DNS record when the DNS request is sent by the Agent to the Client.
• In the error case, delete the DNS record when the timer expires in order to recover.
• In the working case, reset the expiration time to the TTL received in the DNS data.

Workaround: Manually clear the DNS cache or manually query the "stuck" FQDN.

SBX-117469

(11.0.0)

Azure: Upgrade is failing from 9.2.3R4 build to 11.0.0 on standalone.

Impact: Upgrades are failing in Azure using Ribbon IaC package.

Root Cause: Newer versions of the Azure CLI change the way it authenticates, meaning it no longer works with the Ribbon IaC package.

Steps to Replicate: 

1. Log in to the Azure CLI.
2. Attempt to upgrade the Azure SBC using Ribbon IaC.
3. The upgrade fails due to lack of credentials.

The code is modified to install Azure CLI version 2.24.0

Workaround: 

1. Uninstall the current Azure CLI version.
2. Install the Azure CLI version 2.24.0 using the system package manager.

SBX-118683

(9.2.1)

The active T-SBC switches over to the standby T-SBC with a health check failure.

Impact: The SWe UXPAD core-dumps due to an arithmetic exception.

Root Cause: 

1. The arithmetic exception is the result of a denominator of a divide operation with a zero value.
2. The denominator is a sum of two energy values, both of which are expected to be positive. The issue occurs when one of the energy values is negative.
3. The negative energy value is attributed to the incorrect handling of overflow when computing the energy value. The incorrect handling of the overflow makes a large positive number appear to have a negative value.

Steps to Replicate: The issue is not reproducible.

The code is modified by setting the energy value to the maximum positive value if an overflow occurs.

Workaround: None.

SBX-117762

(10.1.3)

If there is only one m-line in the SDP of UPDATE with port ZERO, the SBC is having different behavior for Audio, Video and Application call flows

Impact:
Ingress is sending UPDATE and setting Media Port to ZERO.

1> For m-line application
The SBC is rejecting this UPDATE with 488.

2> For m-line Audio
The SBC is passing this UPDATE to other side with valid port and c-line is present.

3> For m-line Video
The SBC is passing UPDATE to other side but removing C-line

The SBC must pass an update for m-line application and also c-line should be present in all the cases.

Root Cause: The code was written to reject update when media is application and port is zero.

Steps to Replicate: Send update with m-line application, video and audio with port as zero. The update should pass to the other side and C-line should be present in all the cases and port should be zero.

The code is modified to allow the call flow to function accordingly after the fix:

1> For m-line application
The SBC is passing this UPDATE to the other side with valid c-line.

2> For m-line Audio
The SBC is passing this UPDATE to other side and c-line is present.

3> For m-line Video
The SBC is passing this UPDATE to other side without removing C-line.

In all the above scenarios if UPDATE is sent from ingress side with port zero, it is received on the other side with port zero.

Workaround: The call flow after the fix:

1> For m-line application
The SBC is passing this UPDATE to the other side with valid c-line.

2> For m-line Audio
The SBC is passing this UPDATE to other side and c-line is present.

3> For m-line Video
The SBC is passing this UPDATE to other side without removing C-line.

In all the above scenarios if UPDATE is sent from ingress side with port zero, it is received on the other side with port zero.

SBX-118425

(9.2.4)

The SBC as a TLS server fails a TLS handshake attempt if an ECDHE cipher is used and the TLS client specifies an elliptic curve other than secp256r1 (prime256v1)

Impact: The SBC as a TLS server fails a TLS handshake attempt if an ECDHE cipher is used and the TLS client specifies an elliptic curve name other than secp256r1 (prime256v1).

Root Cause: The SBC uses the default elliptic curve name secp256r1 (prime256v1) when acting as a TLS server.

Steps to Replicate: Use "openssl s_client" to trigger TLS handshake with ECDHE ciphers with curve names secp384r1 and secp521r1.

The following commands assume the SBC's SIP-TLS address/port number is 10.x.yy.zz:5061:
$ openssl s_client -showcerts -connect 10.x.yy.zz:5061 -cipher 'ECDHE-RSA-AES128-GCM-SHA256' -curves secp384r1 | egrep "i:\s:"
$ openssl s_client -showcerts -connect 10.x.yy.zz:5061 -cipher 'ECDHE-RSA-AES128-GCM-SHA256' -curves secp521r1 | egrep "i:\s:"

A. Failed case shows the the SBC TLS server fails to find matching cipher suite and curve name:
$ openssl s_client -showcerts -connect 10.x.yy.zz:5061 -cipher 'ECDHE-RSA-AES128-GCM-SHA256' -curves secp384r1 | egrep "i:\s:"
140485529323328:error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure:ssl/record/rec_layer_s3.c:1544:SSL alert number 40
...
Packet capture will not show Server Hello Message.

B. Successful case displays the SBC TLS server's certificate chain after TLS server properly selects cipher suite and curve name.
$ openssl s_client -showcerts -connect 10.x.yy.zz:5061 -cipher 'ECDHE-RSA-AES128-GCM-SHA256' -curves secp521r1 | egrep "i:\s:"
depth=2 C = US, O = Sonus, OU = SonusProductPKI, CN = SonusProductRootCA
...
The packet capture will show Server Hello after the SBC TLS server selects the matching cipher suite and curve name.

The code is modified to allow the SBC TLS server to select the Elliptic curve name automatically.

Workaround: Do not specify the curve name, or use curve name prime256v1 (openssl 1.0's default on TLS server) when using a tls_ecdhe* cipher suite.

SBX-116997

(8.2.6)

Late media pass-through calls fail after an upgrade from 8.2.2R0 to 8.2.6R0.

Impact: Late media pass-through calls with "RTCP termination for passthrough" flag enabled failed.

Root Cause: There are two receiver IDs in the RTCP termination Bus RESource definition. If one of the RIDs was not enabled during the BRES activation because the ingress XRES (ethernet resource) had not learned the destination MAC address yet, then the RTCP Gen was left enabled according to the design.

When that BRES gets deactivated, we skip sending RID disable command to NP for that not enabled RID, and also missed issuing RTCP Gen disable command to NP.

So when that BRES gets re-activated, the logic detected that RTCP Gen has already been enabled and failed the activation process, which then fails the call.

Steps to Replicate: Establish a late media pass-through call with "RTCP termination for pass-through" flag enabled.

The SBC sends a SIP BYE once it receives the SDP answer in the SIP ACK SDP.

The code is modified to send the RTCP Gen disable command to NP even when RID is not enabled.

Workaround: Turn off the "RTCP termination for pass-through" flag.

SBX-113233

(9.2.1)

Getting bulk alarms of "sonusSbxNodeResourcesNoPacketsReceivedNotification"

Impact: The patch includes additional stats from SWe_NP, which can help in triaging false reporting of RTP inactivity issues better.

In addition to this, the patch also includes fixes to prevent false reporting of RTP inactivity when policer mode is set to “Bypass” (from release 9.2.3), and preventing false RTP inactivity reporting in “call hold and unhold scenario” (from release 10.1.2R0).

Root Cause: There is no root cause identified.

Steps to Replicate: The steps are not reproducible. 

The code is modified to add additional NP stats to help in triaging false reporting of RTP inactivity, and also backported some known false detection fixes.

Workaround: None.

SBX-118331

(10.1.2)

EMS re-Synchronization feature not working fine with SBC 7000

Impact: EMS trap resync logic does not work for standard SNMP traps e.g linkUp, linkDown.

Root Cause: The EMA was not aware of the mapping from the standard OID traps to trap name and as a result it was unable to resync trap information to EMS.

Steps to Replicate: 

1. Bring down mgt1 interface on the SBC side, critical alarm raised on the SBC and same has been reached to EMS.
   ip l set mgt1 down
2. Teardown the connectivity between EMS & SBC by creating the static route at SBC side (no trap will be reached to EMS).
3. Bring the mgt1 interface up on SBC side. (Clear trap generated on SBC but not reached to EMS because link between EMS & SBC is down so which is expected).
4. Bring the connectivity between EMS & SBC up (trap re-sync should work and EMS should clear the critical alarm raised on Step 1).

The code is modified so that the EMA now understands the OID values for the standard SNMP traps and can resync the info to the EMS.

Workaround: None.