SBC Core 08.02.04R000 Release Notes

The SBC crashed and a coredump occured.

Impact: The SCM process coredumps due to NULL pointer access on a pointer that has been freed due to BYE and HOLD race in a transfer call scenario.

Root Cause: There was an illegal memory access, absence of NULL check, exposed due to race condition.

Steps to Replicate: A calls B, B REFERs to C and now A and C talk. After sometime, A sends BYE and C sends re-INVITE with a=inactive at the same time.

The code is modified to address the issue.

Workaround: No workaround.

A memory leak was observed in SAM process.

Impact: A memory leak was observed in the SAM process.

Root Cause: There is race condition in the code that handles CALL_AUDITs and CALL_CLEANUPs that can cause a memory leak.

If the response to a CALL_AUDIT takes to long, the code that handles the "late" response has allocates memory for a structure that may never get freed.

Steps to Replicate: This issue is caused by a race condition that cannot be forced and therefore we cannot specify steps to reproduce this issue.

The code is modified to add a timer every time a fault structure is allocated. When the timer expires, if the the structure still exists it is freed.

Workaround: N/A

Portfix SBX-95983: The Customer SBC was seeing STEAL_WARNING:

Impact: Getting a steal warning at login.

Root Cause: The filtering criteria for a high steal value was not well defined.

Steps to Replicate: The script runs as a cron job. This can also be manually tested by making changes in mpstat logs.

The code is modified to detect and warn about high %steal value.

Workaround: N/A

Portfix SBX-103629: The SIP registrations are failing. The SBC was reporting REGISTER_PARSE_ERROR and replies with SIP 500 when it receives retransmitted initial REGISTER (CSEQ 1), followed by REGISTER with Authorization header (CSEQ 1).

Impact: When the SBC recv rexmit of registration (cseq=1) after sending a 401/407, the SBC relays to the AS. At the same time, the IAD sends new registration(cseq=2) for authentication. It triggers a race condition to the SBC and response 500.

Root Cause: The SBC should relay rexmit (cseq=1) registration to the application server.

Steps to Replicate: The IAD send registration(cseq1) to AS, AS response 401 to IAD, IAD rexmit the same sceq1, and send a new sceq2 with auth header.

The code is modified to response rexmit request (cseq=1). As a result, a subsequent one cseq=2 can properly handle the relay to AS.

Workaround: N/A

SIP calls stop when receiving a 183 (Precondition).

Impact: A call stopped working after a 183 received without SDP.

Root Cause: As part of egress precondition interworking, to negotiate the SBC should send an UPDATE to egress endpoint. When 183 received without SDP, the SBC should not try to send UPDATE. The SIPSG call state assumed an UPDATE is sent. As a result, when the next 183 is received it tried to queue the 183 assuming UPDATE is in progress.

Steps to Replicate: 

1. Configure the SBC for precondition egress interworking call flow.
2. Send an initial INVITE from the ingress without precondition attributes.
3. The INVITE is sent with preconditions to egress endpoint.
4. (1) INVITE ====>
   (2) <======= 100
   (3) <======= 181 (without Require)
5. Send the first 183 Session progress without SDP
   (4) <======= 183 (without Require)
6. Send the second 183 Session progress with SDP and precondition attributes.
   (5) <======= 183 (Require: 100rel,precondition)

The above 183 is not processed due to bug in the code. As a result, retransmissions are observed.
(6) <======= 183 ((5) is re-sent)
(7) <======= 183 ((5) is re-sent)

The code is modified to avoid this state transition when 183 received without SDP.

Workaround: Remove the 183 without SDP message using SMM.

Unable to add or Edit route with # in Destination National - CLI or EMA.

Impact: User is not able to store some special characters for the destination number field in the route entity.

Root Cause: The pattern defined for the destination number field did not allow all special characters.

Steps to Replicate: Create a route with special character like #123 for the destination number field in the route entity. It should be successful after this fix.

Portfix SBX-102833: MS Teams calls are dropping when placed on hold.

Impact: When MS Teams puts a call on hold and then attempts to retrieve it while the microphone is disabled, this results in the call being cleared.

Root Cause: MS Teams actions the call retrieve by sending an INVITE with replaces message to the SBC. When the microphone is disabled, this message includes a=recvonly.

The SBC sends a 200 Ok response to the INVITE with replaces with a=sendrecv, while MS Teams is expecting a=sendonly. This causes MS Teams to clear the call.

Steps to Replicate: 

1. Establish a PSTN to MS Teams call through the SBC.
2. Place call on hold from MS Teams.
3. Disable the microphone in the browser running MS Teams, and then retrieve (place off hold) the call.
4. The call is dropped.

The code is modified to respond to an INVITE with replaces containing a=recvonly with 200 OK containing a=sendonly.

Workaround: N/A

Portfix SBX-90251: An SM process healthcheck timeout occurred during a configuration restore on the Cloud SBC.

Impact: The SM process crashes due to a healthcheck failure.

Root Cause: As part of a load configuration, the configuration tar file needs to be untared. The untaring is taking longer than the healthcheck time out, when the size is big.

Steps to Replicate: 

1. Save the configuration.
2. Load the configuration.
3. Check if healthcheck failed and the SM crashed.
4. The SM process should not have crashed and the load configuration must have been completed succesfully.

The code is modified for the running tar command so that the healthcheck does not fail during a load configuration.

Workaround: Override the healthcheck for the tar command.

Portfix SBX-103207: The SBC SWe 8.2.0F1 duplicates audio from call B to call A.

Impact: If CN is not negotiated for G711 codec and remote peer still sends CN packets that match default CN payload type (13), then user on other end may hear cross talk audio of completely unrelated channel or hear own reflect audio.

Root Cause: When a g711 side does not negotiate CN in signaling, DSP does not initialize Comfort Noise Generation object. However, if remote peer still sends CN packet that matches the g711 SID payload type configured in PSP (default 13), DSP accepts the packet. It processes that CN packet incorrectly and as a result uses stale voice buffer which happens to be of another channel. This continues until next voice packet for that channel arrives. Hence, we happen to see cross talk audio from other call during silence period. In some cases, the user may hear own audio also and that is different manifestation of the same problem. This issue is specific to SWe.

Steps to Replicate: 

1. Run the CN in signaling.
2. The DSP will not initialize Comfort Noise Generation object.
3. Send a remote peer to the CN packet that matches the g711 SID payload type configured in PSP (default 13) and the DSP accepts the packet.
4. Process that CN packet and as a result, uses stale voice buffer that happens to be of another channel.
5. This continues until next voice packet for that channel arrives.

The code is modified to initialize the comfort noise object even though CN is not negotiated and process the CN packets correctly.

Workaround: There are multiple workarounds for this issue:

1. Change the default payload type of comfort noise from 13 to something else (say 15) in PSP of peer that is sending CN packets. This will make DSP drop CN packets because payload type will not match. Run the PSP configurationshow

configuration profiles media packetServiceProfile G711_NONE_NONE_NONE..silenceInsertionDescriptor {g711SidRtpPayloadType 13;heartbeat enable;}

2. Enable the silence suppression on PSP of peer that is sending CN packets and keep CN payload type that matches with CN payload type.

SBX-94491

1

It takes 7 seconds for the standby server to detect a switchover triggered by a reboot thus delaying to time to become active (OpenStack 1:1 HA).

Impact: A switchover triggered by the reboot takes 7 seconds to be detected by the standby, increasing the switchover time and causing media takeover issues.

Root Cause: The transport protocol is responsible for noticing the peer has gone away, and with the UDP protocol used for SWe, this process takes too long.

Steps to Replicate:This does not explain the steps to take to replicate the issue. 

The code is modified to alert the peer to take over rather than relying on the transport protocol health check to timeout.

Workaround: None.

Portfix SBX-101628: The CPX process leaks memory for a call load.

Impact: There was a memory leak in MsgClient message queue.

Root Cause: The dispatcherAgent's MsgClient was not instantiated properly, causing messages to be queued and never deleted.

Steps to Replicate: Perform traffic test with ASAN load to confirm that memory leak is no longer present.

Instantiate DispatcherAgent's MsgClient properly and to not use the message queue. Run a clear message queue in destructor of MsgClient to address the issue.

Workaround: None.

Portfix SBX-102737: The SM Process experiences a core dump on both the active and standby after an sbxrestart.

Impact: The SM Process cores at startup due to healthcheck.

Root Cause: System_i::fillSavedConfigurations is trying to read /mnt/gfsvol1/config-versions.txt, but it is taking too long.

Steps to Replicate: 

1. Set up the MRFP system with OAM.
2. Perform an upgrade.
3. Perform a system restart.

Pause healthcheck in System_i::fillSavedConfigurations to address the issue.

Workaround: Reboot the failing node.

Portfix SBX-91476: The PEM header contains a gated parameter.

Impact: The SBC is sending a gated parameter in the PEM sendrecv toward the Ingress
P-Early-Media: sendrecv gated.

Root Cause: By design if the Early Media is gated by a peer or gated locally, the SBC adds the gated parameter towards the Ingress.

Steps to Replicate: 

1. On the Egress Leg, the earlyMedia method should be pEarlyMedia.
2. An INVITE should be received with the P-Early-Media: supported. The tone criteria is configured to play tone with 183 message.
3. The 183 is recvd with the P-Early-Media: inactive.
4. The SBC starts playing the tone as tone criteria matches.
5. The SBC sends a P-Early-Media: sendrecv,gated toward Ingress in 183.

A new flag "suppressGatedParam" is added at the TG level.
If this flag is enabled, the SBC does not add gated param towards the Ingress.

set addressContext default zone <ZONE_INGRESS> sipTrunkGroup <TG_INGRESS>media earlyMedia suppressGatedParam <enabled/disabled>

Workaround: N/A

Portfix SBX-103948: The SBC Application is crashing when a CPaaS to PSTN call is made.

Impact: After a reboot/restart, the CPX process could not read the sdpContent values from "action/from" and "action/to" SMM definitions, as a result while applying the action during a call it does not find the sdpContent and causes a core to happen due to processing a null or incorrect value.

Root Cause: After a reboot/restart, the CPX process is unable to read/action/from/sdpContent and /action/to/sdpContent due to wrong path/invalid pointer.

Steps to Replicate: Configure the SMM rules to delete the SDP line and generate traffic to confirm it works correctly.

The code is modified to read and store the from/sdpContent and to/sdpContent correctly in the SCM Process cache.

Workaround: Disable the SMM that deletes the SDP.

Portfix SBX-101765: The SBC is going down when running NNIProfile CLI

Impact: When the application code was reading the NNI profile from CDB, there was a memory overwrite problem.

Root Cause: The local variables that were used to store the information being read from the CDB were not large enough resulting in memory being overwritten.

Steps to Replicate: Replicate in lab environment only.

The code is modified to use large local variables to avoid the memory overwrite.

Workaround: None.

The SipSignalingPorts OutOfService after switchover.

Impact: The SIP sigPorts stuck in OOS after a switchover.

Root Cause: Customer had the network/pkt port issue, on one of their HA node, which caused pkt port(s) to bounce randomly, i.e. pkt port went DOWN and came back UP within 2 to 3 seconds. So they have been keeping that node as standby node. They also have link detection enabled for pkt port(s) and have around 100 LIFs per pkt port, one per SIP SIGPORT. When pkt port went down, NRS delays the port down event processing for 2 second to allow link failure detection to be ready and also to avoid the race condition between NRS and LVM. When 2 second delay timer is up, NRS starts to take down affected LIFs and notifies local SIPCM and SIPFE so they can take down affected SIP SIGPORTs.

In SIPCM, all the sockets on the affected sigPort are put in a delete pending table and starts a 1 tick timer. Then the socket(s) is being deleted after 1 tick timer is up.
When pkt port came back up, NRS processes the event with no delay and notifies SIPCM and SIPFE as well. Since there are around 100 LIFs, there were many messages exchanges between NRS/XRM/SIPCM/SIPFE. NRS LIF FSM has the mechanism in place to handle the timing issue and LIFs were all back in service. But SIPCM failed to activate some SIP SIGPORT(s) while binding the socket(s). These error messages indicated that SIPCM tried to activate the SIGPORT while it was still pending delete. Therefore SIGPORT got stuck in OOS(broken state) in both SIPCM and SIPFE on standby node. If there was a switchover happened later, user would then noticed one or more SIGPORTs were OOS. They have to manually bounce those SIGPORTs to bring them back in service.

Steps to Replicate: The nature of the problem was the timing caused race condition. There is no good way to re-create/verify the fixes.

The code is modified in the following manner:

1. Introduced a new 1-tick timer in SIPCM_DATA_STR, activateRetryTimerId.
2. In SipCmActivateCallSigPort(), added deletePendingSocketTable so that if sigPort is found in the table, then the 1-tick timer starts. When the timer is up, SipCmActivateCallSigPort() is invoked again. Once the sigPort is activated successfully, SIPCM notifies SIPFE as usual.

Workaround: None.

Portfix SBX-101922: GUI CDR Configuration screen - Syslog server field can not be configured.

Impact: The server section is appearing on the CDR configuration, but the configuration does not participate in configuration.

Root Cause: The child object was added. As default behavior of the EMA, the added child object was not needed.

Steps to Replicate: Log in to the EMA and configure the CDR and ensure the Syslog configuration is not listed on the CDR configuration page.
configuration--> system provisioning --> Base provisioning --> CDR configuration.

Restricted the child object in this particular case to address the issue.

Workaround: None.

The RE-REGISTAR was shorter than 3600 secs due to a child AoR. 

Impact: When the 200 OK response to REGISTER request contains a P-Associated-URI header, the SBC creates a child AOR and forwards all REGISTER refresh requests to the registrar (effectively disabling the SBC fast refresh response).

Root Cause: The creation of child AORs cause all REGISTER refresh requests to be sent to the registrar (effectively disabling the SBC fast refresh response).

Steps to Replicate: 

1. Configure the SBC to perform registration.
2. The 200 OK response to REGISTER request contains a P-ASSOCIATED-URI header.
3. In the error case, the refresh REGISTER requests are all forwarded to the registrar.

Removed the child AOR check from SipRaRegisterRequestCompletedAorNfy(), permitting the SBC to send fast refresh responses.

Workaround: N/A

Portfix SBX-103645: There was a a customer 81.R5 upgrade with a spiltbrain M-SBC boot issue.

Impact: One of the M-SBCs in N:1 mode is not coming up after an upgrade.

Root Cause: The M-SBC node went for multiple reboots due to a configuration profile change and split brains after an upgrade exceeding the max limit for reboot.

Steps to Replicate: Upgrade all M-SBC nodes at one time so that they all come up together.

The code is modified to avoid split brain due to nodeId/serviceId collision by informing the peer nodes about the self node's nodeId/serviceid as soon as it is allocated.

Workaround: Restart the M-SBC application manually on the node that has exceeded max reboot limit.

Portfix SBX-98243: The SIPREC Status command is not showing all four SRS servers after upgrade completes

Impact: The SIPREC status does not correctly display all SIPREC servers if the SBC does a SYNC on an HA set up when more than one SIPREC recording is active. The SYNC occurs when the standby box is restarted while the active remains up. This is not an issue if a new SIPREC call is started while the HA set up is already running in active/standby mode. This problem only impacts the status screen and not the processing of the SIPREC call.

Root Cause: After a SYNC, only one SIPREC entry is maintained in the SIPREC status on the standby server. So if a switchover then occurs, the status screen only shows one active call, although all the other active SIPREC calls are maintained correctly.

Steps to Replicate: 

1. The HA Set up with the SIPREC (Quad or single).
   If the Quad Recording, single call is sufficient.
   If there is a Single recording, there is multiple calls with SIPREC.
2. Trigger the SYNC (Restart the standby box).
3. After SYNC is complete, switch over.
4. Check the status of the recordings.

The code is modified to maintain the status information of all the SIPREC calls from the active server.

Workaround: Do not restart the standby server while there are SIPREC calls running on the active server.

The SBC is rejecting the second Update message from the Egress as a 500 result into a call failure.

Impact: The SBC rejects the second Update from the egress due to a DLRB feature.

Root Cause: The First Update SIPS response locally 200OK but not clear the server request message. As a result, the second Update triggers a SIPS answer 500.

Steps to Replicate: Configure the DLRB, the Egress response 183 with the SDP, the first Update, and the second Update.

The code is modified so after a response 200OK, the SIPS clears the server request so it handles the subsequent one.

Workaround: Disable the DLRB.

A DSP core occurred on the SBC.

Impact: The DSP faults with a memory fault in area that pulls buffers out of EMAC port. Suspicion is on the buffer size that is reported by EMAC buffer. These errors result in a DSP reload which is essentially ensures call continuation. Any calls which are using this DSP could have a small media outage while the DSP is reloaded.

Root Cause: The root cause is a speculation that the EMAC system is occasionally is a incorrect bufSize, similar to a bit flip.

Steps to Replicate: This issue cannot be reproduced, it is fixed based on code review and coredump backtrace.

The code is modified to look for consistency in packet size and EMAC buffer size reported. In case an inconsistency is found, the packet is rejected, avoiding illegal memory access.

Workaround: None.

Portfix SBX-101394: The SBC restarts after receiving 3xx with the embedded Identity in Contact header.

Impact: Global-buffer-overflow error when the SBC received a 3xx with the Identity header embedded in the contact header.

Root Cause: The root cause here is that the number of elements in pstKnownTypes array are less than the length on that the loop was running on.

Steps to Replicate: 

1. The UE1 sends an Invite to the SBC.
2. The SBC sends an Invite to the customer UAS.
3. The UAS-1 sends a 3xx with the Identity header embedded in the contact header.
4. The SBC sends an ACK.

Honor Embedded Headers and Enhanced Local redirection flag are enabled in the egress TG's IPSP.

The code is modified to use the actual length of pstKnowTypes array rather than using the SIP_HEADER_MAX. Calling the SipHdrTranspGetNumKnownTypes() function returns the actual number of elements in pstKnownTypes array.

Workaround: None.

Portfix SBX-99134: There was a SCM process coredump after a second switchover while establishing the SIP-GW-H323 call load.

Impact: During a switchover and switchback while running a load, there may be some stale entries in the SIPSG and CC control blocks. Such entries are cleared on an audit but while clearing, the SIPSG is sending CC_SG_UPD_ACCT_NFY_MSG_STR to the CC. Since the load is running a corresponding entry for the CC, the index may be allocated to a new call. As a result, the msgPtr-> gcid is not matching with the ccbPtr->gcid and the gcid stored in the SIPSG.

When the CC receives this message, the mismatch happens and it logs a SYS_ERR(SE_EVLOG, SE_CC_UNEXPECTED_DATA))

Root Cause: While performing a switchover during a high load run, the SBC should be run in normal mode instead of sensitive mode.

Steps to Replicate: In this scenario, the HA pair and a standalone SBC are used.

1. Use the External PSX to establish SIP->GW calls on the active SBC and GW->SIP calls on a stand alone SBC.
2. Configure the IPv4 GW Trunkgroup between an active SBC and a stand alone SBC. Originate the IPv4 SIP->GW call load on an active SBC.
3. Terminate the IPv4 SIP call load on a stand alone SBC to handle the GW->SIP signaling call load.
4. Successfully establish the required call load (28000 sessions).
5. Restart the active SBC.
6. After restart, successfully establish a required call load (28000 sessions).
7. Then perform a second restart on the currently active SBC.
   The default active SBC should not go down.

The code is modified to not use a sys error if the GCID sent from the SIPSG in a msgPtr is different from the GCID in the ccbPtr during a switchover.

Workaround: N/A

Portfix SBX-99600: After a switchover on SBC 5400 platform, the SCM process is coredumping, due to hitting a SYS_ERR while running in sensitive mode.

Impact: When performing a switchover and switchback while running a load, there may be some stale entries in SIPSG and CC control blocks. Such entries are cleared during an audit, but while clearing these entries, the CC is sending SG_CC_DISC_RSP_MSG to SIPSG. Since the load is running a corresponding entry of the CC , the index might be allocated to a new call. As a result of this issue, the msgPtr-> gcid is not matching with the ccbPtr->gcid in SIPSG.

When the CC receives a message, the mismatch happens and the CC logs the SYS_ERR(SE_EVLOG, SE_CC_UNEXPECTED_DATA))

Root Cause: While performing a switchover during a high load run, the SBC should be run in normal mode instead of a sensitive mode.

Steps to Replicate: 

1. Establish total sessions of 141K call load on the Active box.
2. Perform multiple switchovers.

Observed SCM process coredump should not be seen.

If the GCID sent from the CCin msgPtr is different from the GCID in ccbPtr during the case of a switchover, do not use a sys error to address the issue.

Workaround: N/A

Portfix SBX-102472: ImPr Process core dumped for Default LI with IPSEC.

Impact: When a call is intercepted during switchover, the SBC throws SYS error SE_SYS_HASH_EXISTS and causes coredump.

Root Cause: If a call is intercepted during switchover, the standby SBC may have the duplicate info of the intercepted call. So it throws SYS error "SE_SYS_HASH_EXISTS"
The coredump is due to the above SYS error.

Steps to Replicate: 

1. The SBC is configured for Default LI.
2. Run a load of 5 cps of intercepted calls and trigger switchover on the primary SBC.
3. The secondary SBC comes up as active but the primary SBC dumps core.

The fix is given to handle the duplicate call data of the intercepted call on the Standby SBC.

Workaround: Disable the coredump level.

Portfix SBX-97424: The SCM process core observed in the M-SBC after a switchover with the SIPREC.

Impact: The SCM Process coredump is observed in the M-SBC after a switchover when the SIPRec occurred with a stable call and BYE for the main call is triggered post switch-over.

Root Cause: The recorderType field (field describing the type of recording happening in the SBC) is not mirrored. Post switch-over once the main call is terminated, the SIPREC delete command goes from an S node to M node but this does not carry the recorderType field. When the command reaches M node from S node with recorderType set to zero, the M node accesses a NULL pointer (call data channel).

Steps to Replicate: 

1. Establish a stable SIPRec call.
2. Perform a switch-over.
3. Terminate the main call.

Send a recorderType for all type of monitor commands. Add a Call Data Channel NULL check in the M node while processing a MonitorCommands.

Workaround: N/A

The DSP Threshold setting is not generating a trap on SBC 5400.

Impact: The g711PacketThreshold, g729Threshold, and g726Threshold onset and abate traps are not sent.

Root Cause: The NRM did not receive CLI updates to the g711PacketThreshold, g729Threshold, and g726Threshold, and the trap generation code used wrong trap names.

Steps to Replicate: 

1. Provision the threshold levels:
   set oam traps dspAdmin dspAvailabilityTrap g729Threshold 40
   set oam traps dspAdmin dspAvailabilityTrap g726Threshold 60
   set oam traps dspAdmin dspAvailabilityTrap g711PacketThreshold 80
   commit set oam traps admin sonusSbxDspAvailG729OnSetCrossThresholdNotification state enabled
   commit
   set oam traps admin sonusSbxDspAvailG729AbateCrossThresholdNotification state enabled
   commit
   set oam traps admin sonusSbxDspAvailG726AbateCrossThresholdNotification state enabled
   commit
   set oam traps admin sonusSbxDspAvailG726OnSetCrossThresholdNotification state enabled
   commit
   set oam traps admin sonusSbxDspAvailG711OnSetCrossThresholdNotification state enabled
   commit
   set oam traps admin sonusSbxDspAvailG711PacketAbateCrossThresholdNotification state enabled
   commit
2. Configure the trap target: set oam snmp trapTarget EMS160 ipAddress 10.xxx.xx.xxx port 162 trapType v2 state enabled commit
3. Limit the compression resources to make the issue readily occur: set system mediaProfile tone 98 compression 2 commit
4. Perform transcoded (e.g. G711 to G729) calls that exceed threshold limits, and verify that onset traps are sent to the trap target.
5. Clear the transcoded calls, and see abate traps are sent to the trap target.

The code is modified to support g711PacketThreshold, g729Threshold, and g726Threshold values.

Deprecated the support for allThreshold, as it was never implemented in the SBC.

Workaround: N/A

The code is modified to disable the Path MTU Discovery by setting kernel parameter net.ipv4.ip_no_pmtu_disc to 2. When set to 2, the DF bit is set to 0 for the transmitted packets on every created socket. The configuration parameter is "system admin <name of system> kernelParams ipNoPmtuDisc".

Workaround: The workaround is a hack. It can be used as a one time test only. The hack will not survive Linux reboots.
1. Make changes on the current Standby (assuming SBC1 is currently active, and SBC2 is currently standby)
[root@SBC2 ~]# date; cat /proc/sys/net/ipv4/ip_no_pmtu_disc
[root@SBC2 ~]# date; echo “2” > cat /proc/sys/net/ipv4/ip_no_pmtu_disc
[root@SBC2 ~]# date; sbxrestart
2. Wait until the SBC is up and synched. Ensure the p_no_pmtu_disc is set to 2.
[root@SBC2 ~]# date; sbxstatus | tail -4
[root@SBC2 ~]# cat /proc/sys/net/ipv4/ip_no_pmtu_disc
2
[root@SBC2 ~]#
3. Make changes on the current Active.
[root@SBC1 ~]# date; cat /proc/sys/net/ipv4/ip_no_pmtu_disc
[root@SBC1 ~]# date; echo “2” > cat /proc/sys/net/ipv4/ip_no_pmtu_disc
[root@SBC1 ~]# date; sbxrestart
4. Wait until the SBC is up and synched. Ensure the p_no_pmtu_disc is set to 2.
[root@SBC1 ~]# date; sbxstatus | tail -4
[root@SBC1 ~]# cat /proc/sys/net/ipv4/ip_no_pmtu_disc
2
[root@SBC1 ~]#
5. May perform a switchover to make the SBC to be active.
6. Run customer tests.

The display-name in the Diversion header or History-info header is deleted during interworking.

Impact: The SBC does not send display name present in ingress INVITE's diversion header in history info header in egress INVITE.

The SBC does not send display name present in ingress INVITE's history info header in egress INVITE's diversion header.

Root Cause: The SBC does not consider display name when interworking between the diversion header and history info header.

Steps to Replicate: 

1. Ensure the display name is sent in history info header of egress INVITE when the SBC is converting diversion header to history info header.
2. The SBC should send display name in diversion header of egress INVITE when the SBC is converting history info to diversion header.

The code is modified to ensure the display name is sent when the SBC is interworking history info and diversion header.

Workaround: None.

Portfix SBX-103492: Performance: Observed "Scm" Process core on standby box while running the load on Openstack ISBC HA.

Impact: The SCM process cores after a switchover on Standby (new active) node when running the TCP-TCP load at 100 CPS with 10 secs call hold time.

Root Cause: The issue is due to not setting the "prtSigZoneEntry->nextMnsPortPtr" when the SIP signaling port is removed.

Steps to Replicate: Perform switchover when running TCP-TCP load at 100 CPS with 10 secs CHT.

The code is modified so whenever the SIP signaling port is removed, set the prtSigZoneEntry->nextMnsPortPtr to NULL.

Workaround: None.

Portfix SBX-104494: The PES process cored while testing the Flex AD support with ERE.

Impact: The PES process dumps core while executing an Active Directory (AD) service when no AD profile and/or AD attribute profile configured.

Root Cause: While executing the AD service, PES fetches the AD profile from cache. Since there is no AD profile was configured in this case, cache shall return NULL. The code was missing to check for NULL for the AD profile pointer and dereferencing the NULL pointer results in a coredump.

Steps to Replicate: Configure a AD number translation criteria with ingress TG as the trigger criteria type, but with no AD profile and/or AD attribute profile configured. Make a call on the ingress TG.

The code is modified to check for NULL for an AD profile and AD attribute profile. If there is no profiles configured, the control returns and the service is marked as failed.

Workaround: Configure an AD Profile and AD attribute profile so that the existing code can be executed further.

Portfix SBX-104773: The SBC is not coming up on the VMWare when using two NUMAs.

Impact: The SBC fails to come up in the VMWare dual NUMA configuration. (cps service failed to come up)

Root Cause: As part of the CPS service, there is one init script that calculates the performance indices. In the VMWare dual NUMA scenario, the index calculator binary gets launched in the last core, which happens to be on the 2nd NUMA. Since the huge pages are reserved on NUMA0, so the index calculator binary fails (saying fails to create mbuf pool). As a result, the CPS fails to come up.

Steps to Replicate: 

1. Install an ISO in VMWARE dual NUMA instance. Ensure the number of vcpu should be large enough so that we get 2 NUMAs in the instance.
2. Verify if the CPS comes up by "service cps status" command.

The code is modified to always launch on the second core, to ensure the NUMA0 is always used.

Workaround: None.

SM Process core on the Server.

Impact: The SM Process crashed while executing the "show table system syncStatus" command.

Root Cause: The shell script used to get the Oracle sync status - PolicyDBSyncStatus.sh - did not return within 10 seconds, causing a healthcheck timeout that caused the coredump.

Steps to Replicate: This problem is not reproducible.

Disable healthchecks while fetching the syncStatus to address the issue.

Workaround: None.

The SBC running version 6.2.2 cannot capture the media with MCT.

Impact: The MCT reload configuration (SBC restart or switchover) will not succeed.

Root Cause: Logic was missing during the restart in order to properly reload the MCT configuration.

Steps to Replicate: The steps are not easily producible. 

The code is modified to properly reload the MCT configuration.

Workaround: User can delete the MCT config and reconfigure it after any restart/switchover.

The PRS process cored, backtrace shows the PrsNpCoreStatusProc(), and the XSC core is spinning.

Impact: Only one customer reported this peculiar issue where the Network Processor (NP) hangs due to an interrupt generated by the internal hardware module because a zero length packet request was sent.
This caused the module to lock up and NP is no longer able to transmit any packets.

Root Cause: We were never able to find anything in the code that would send a packet with zero length.

Steps to Replicate: You really cannot test this change without damaging the Network Processor.
The zero-length check was tested by adding debug code that would force a zero-length check and handle the interrupt.

The code is modified to log the event and not submit the zero length packet to the hardware module. The added code also handles the interrupt and continue running.

Workaround: None.

The SM process and CPX cores prevented the server to come back up as standby after switchover.

Impact: The SM process and CPX processes cored on slot 1 after LDG triggered switchover to slot 2.
The SM process was deadlocked while retrieving peer NTP status.
The CPX cores happened after SM process coredump, when writing SMM profiles into shared memory.

Root Cause: The root cause was SM process deadlock.

Steps to Replicate: This is time sensitive issue. I do not have steps to re-produce or verify the fix.
Suggest to run full regression test with switchovers.

The code is modified to release the smNtpLock_ after issued NTP restart command.

Workaround: None.

Portfix SBX-104769: The SCM process cored while testing the SBX H323 Conformance and Interworking.

Impact: The SCM process will core for the RFC 2833 to Inband DTMF in SIP to H323 interworking call scenario.

Root Cause: Accessing of a NULL pointer caused the core dump.

Steps to Replicate: Run a SIP-H323 interworking call with SIP side rfc2833 and the H323 side Inband DTMF being enabled.

The code is modified to check the NULL pointer before accessing it.

Workaround: None.

The SBC was sending m=application 0 UDP/BFCP (null).

Impact: The SBC generates the syntax error m-line UDP/BFCP in 183.

Root Cause: The SBC accesses initializes data when the format m-line for UDP/BFCP.

Steps to Replicate: An incoming call with the m-line UDP/BFCP. ERE/PSX using script NO_ROUTE and triggers an announcement. The internal 183 has m-line UDP/BFCP with invalid syntax.

Make m-line initialize properly to address the issue.

Workaround: Use the SMM to correct m-line syntax error

Crosstalk issue on the SBC 5400.

Impact: When a transcoded call with Opus codec and the Opus termination does not send any packets to the SBC, then another termination of Opus call may hear the audio from another unrelated Opus-transcoded call or it may hear white noise.

This occurs on the SBC5x10/5400/7K but not on the SWe platforms.

This problem only occurs if no Opus packet is received. When the decoder receives the first Opus packet, then subsequently the problem does not happen for that call.

Root Cause: Initially, when no packet is received, the Opus decoder is not called. As a result the decoder's output buffer is uninitialized and it happened to be a stale buffer of another Opus call or a leftover buffer of a previous Opus call on that DSP. That buffer is used to encode data for other leg of the Opus call and that results in cross talk audio or distorted noise to other termination of Opus call.

Steps to Replicate: 

1. Make two SIPP Opus<=>g711 calls, say call 1 and call 2 on SBC5x10/5400/7K.
2. These two calls have to land on same dsp core. So issue a unhide debug command 'request sbx drm debug command "loadbalance disable" before making two calls. Without this command, both calls may not land on the same DSP core and you may not see the same problem.
3. Call 1 sends audio packet (.pcap) from Opus termination but call 2 does not send any packets to the SBC from its Opus termination.
4. Call 2's g711 termination will hear audio from call 1's Opus termination.

Clear the buffer (with silence) when the Opus decoder is not primed with any packet to address the issue. 

Workaround: 

1. Use a different codec other than Opus.
2. Use SWe platform instead of h/w SBC5x10/5400/7K

Enabling the Dialog Transparency causes calls to fail.

Impact: A call fails for dialog transparency and forking.

Root Cause: When the ingress support prack and multiple 18x fork, the SBC fails to send 200OK to ingress.

Steps to Replicate: 

1. Configure the dialog transparency and downstream forking on egress.
2. The Ingress support prack, egress not support prack.
3. The Egress answer 180fork1, 180fork2, 180fork3, and 200OKfork2 simultaneously.

The code is modified to address the issue.

Workaround: Disable the dialog transparency.

Portfix SBX-103553: The SBC does not allow multiple 183.

Impact: A GSX-GW-SBX call erroneously queues a 183 message on the SBC when the X-Service-Type precondition is received and the downstream forking is enabled.

Root Cause: The interaction between the downstream forking and GW-GW code.

Steps to Replicate:

1. Make a GSX -> SBX SIP-GW-SIP call.
   The Egress trunk group has X-Headers supported.
   The Egress trunk group has downstream forking enabled.
2. INVITE -->
   <-- 183 with X-Service-Type: cf,precondition and SDP
   <-- 183 with X Service-Type: cf,precondition and different SDP
   <-- 180 with X Service-Type: cf and SDP
   <-- 183 with X Service-Type: cf and no SDP
3. The final 183 is queued.

The code is modified to not perform queueing in GW-GW scenarios.

Workaround: None.

The SCM process cored.

Impact: The SCM process cores after long extensive load of an OOD relay.

Root Cause: The SBC accesses an invalid pointer of link list when the cleanup relay control block.

Steps to Replicate: The issue is not easily reproducible.

The code is modified to properly initialize the link list.

Workaround: None.

Portfix SBX-104934: The SBC generates the same ICID for more than 1 call, porting the fix for SBX-101887 into 8.2.3F1 did not help.

Impact: Duplicate the ICID generated for two different calls.

Root Cause: The instance specific value is set to 0 every other call causing the ICID generation to always use 0 in all instances. This results in the same ICID generated in more than one SCM instance when the calls land within the same microsecond.

Steps to Replicate: Configure the SBC to generate ICID on the egress side. Run a call load and check for duplicate ICIDs.

The code is modified to manipulate the last octet of the MAC address used in ICID generation.

Workaround: None.

Portfix SBX-104688: There was a configuration import issue.

Impact: The fields that support \r like regex fields, changes it to \n after configuration Import (XML based).

Root Cause: The \r is not xml encoded when parsing the intermediate xml file.

Steps to Replicate: 

1. Create SMM Profile with regex that has \r.
2. Export the configuration as xml-tar.
3. Clear the DB.
4. Import the configuration.
5. Verify SMM profile regex field.

The code is modified to handle the \r as a special case.

Workaround: None.

The SBC does not select DNS servers in the correct ratio when all weights are set to a value of 1 or 100.

Impact: When multiple DNS Server in a DNS Group are configured with weight = 1 then one of the DNS Sever does not get any traffic.

Root Cause: Off by one error in handling of DNS load distribution logic.

Steps to Replicate: 

1. The SBC is configured with DNS Group which has 2 DNS servers with priority = 0 and weight = 1.
2. Make multiple calls that requires FQDN resolution to reach out DNS Servers.
3. Monitor the DNS server Statistics and note one of the DNS sever is not used.
   show status addressContext default dnsGroup DnsGrp dnsServerStatistics.

If the weight = 1 for all the server SBC does not do a hard round robin for load balancing.

The load balancing depends on the random value generated but it is expected that for a large number of samples the load will be almost evenly distributed between the servers.

The code is modified to handle this scenario.

Workaround: Avoid configuring DNS servers with weight = 1 and recommended to use higher weight values as workaround.

The Qseries CDR field not matching Qseries format.

Impact: When the rn parameter is received as part of SIP R-URI, this value is being used to populate field 10 of the QSBC format CDR record, rather than the received Called Number

Root Cause: In case of rn parameter received, the source for field 10 of QSBC CDR is incorrect.

Steps to Replicate: Send a SIP INVITE with R-URI including rn parameter.

The code is modified so if the rn is received, field 10 of QSBC is populated with the "Called Number Before Translation" (field 24 of Ribbon format CDR). For other scenarios, field 10 is populated as before.

Workaround: None.

The SCM Process core dumped in the SIGABRT.

Impact: The SCM has cored due to a memory corruption issue.

Root Cause: The SCM core is the result of a memory corruption caused by a function that is overwriting the end of the buffer that it allocated.

Steps to Replicate: This issue is not reproducible.

The code is modified to prevent the function from overwriting the end of the buffer it allocated.

Workaround: N/A

Portfix SBX-102082: An SCM coredump observed when the REGISTER is received and the useRandomUserInfoInContactHdr is enabled.

Impact: There was an invalid memory access issue when both embeddedRegInfoinUserPart and useRandomUserInfoInContactHdr flag were enabled.

Root Cause: In the SBC appropriate memory was not allocated for the contactUrl puchUsername in case both embeddedRegInfoinUserPart and useRandomUserInfoInContactHdr flag were enabled, due to this invalid memory access was happening.

Steps to Replicate: 

1. Enable the remoteDeviceType to access.
2. Enable the embeddedRegInfoinUserPart flag as:-
   set addressContext <ADDRESS_CONTEXT> zone <ZONE> sipTrunkGroup <TG> signaling embeddedRegInfoinUserPart enabled
3. Enable the useRandomUserinfoInContactHeader as :-
   set addressContext <ADDRESS_CONTEXT> zone <ZONE> sipTrunkGroup <TG> signaling useRandomUserInfoInContactHdr enabled

The code is modified to allocate the appropriate memory for puchUsername based on embeddedRegInfoinUserPart or useRandomUserInfoInContactHdr enable case.

Workaround: As per the SBC design anyone of the below flags should be enable not both.

The SBC is sending the wrong TO tag to the AS side.

Impact: The response 200OK for multiple Notify's has an invalid ToTag.

Root Cause: There was an internal logical error when processing the second 200OK Notify from AS. The application had a misinterpretation of the direction for the relay message as an IAD and through passing down the wrong data down to SIPS for relaying 200OK to AS.

Steps to Replicate: Subscribe the IAD to AS. The AS sends multiple Notify immediately one after another.
The second 200OK of Notify sends to AS has a wrong ToTag.

The code is modified so that the SBC passes the correct data down to SIPS.

Workaround: The peer can send 1 Notify at a time. N/A on the SBC.

After a cleanDb, a user is unable to login to the KVM SBC as an admin user using keys.

Impact: The datasourceList was set to 'None' for KVM platform. After running the clearDB, the cloud-init was not having any datasource to read the ssh keys.

Root Cause: The change introduced as part of Jira SBX-88682 caused this issue.

Steps to Replicate: Instantiate VM on KVM with ssh keys configured for access to linuxadmin and admin. Perform clearDB and try to login with in ssh keys to admin and linuxadmin.

The code is modified to use 'NoCloud' Data source to address the issue.

Workaround: Remove '/etc/cloud/cloud.cfg.d/99-datasource.cfg' file before doing clearDB.

The SBC fails to relay 4xx-6xx responses when IPTG authentication is enabled after 100 trying

Impact: When IPTG authentication is enabled, the SBC maps all 4xx-6xx SIP codes to CPC 176. As a result, the SBC is unable to relay cause code from egress to ingress endpoint. Example: 486 Busy Cause code was not relayed to ingress side.

Root Cause: When IPTG authentication is enabled, the SBC maps all 4xx-6xx SIP codes to CPC 176.

Steps to Replicate: 

1. Reproduce the issue. With IPTG authentication feature enabled as below, the SBC sends INVITE with Authorization header as response to 407 from egress. However, when the egress sends 486 BUSY as a response to Authorization INVITE, the SBC does not transparently send 486 towards ingress endpoint.
2. Verify the issue. The SBC correctly sent 486 BUSY received from egress to ingress.

The code is modified so the SBC, upon receipt of error code other than 401 and 407 clears the authentication flag and relays the actual cause code from egress to ingress

Workaround: Use CPC mapping to map CPC code 176 to required SIP cause code.

There was a switchover // sonusCpSystemProcessCoredumpGeneratedNotification.

Impact: The SCM process has cored.

Root Cause: The SCM has cored when attempting to write a log message because the code is attempting to de-reference a NULL pointer to access information for the log message.

Steps to Replicate: There are no specific steps for reproducing this issue. This core will only happen in a multi-transfer scenario.

The code is modified to check that the pointer is non-NULL before de-referencing it.

Workaround: There is no workaround. This core will only happen in a multi-transfer scenario.

Filtering does not work on the Cleared Alarm List.

Impact: Filtering does not work on the Cleared Alarm List.

Root Cause: There is an order mismatch before applying the time filter and after applying the filter.

Steps to Replicate: 

1. Log in to EMA.
2. Navigate to Monitoring->Alarms->Cleared Alarms.
3. After that, apply the time filter (from -time -> to- time). Then we are able to see the correct order of time.

The code is modified to address the issue.

Workaround: None.

Portfix SBX-103974: The STI sends 2 Identity fields in Contact header, but SBC only passes one

Impact: In the case where multiple "Identity Headers" are present as embedded headers in a 3xx "Contact Header", the SBC sends out only 1 Identity Header for the re-directed INVITE.

Root Cause: The SBC did not support handling of the scenario where SIP headers are repeated in embedded contact headers in 3xx. The SBC would end up the sending only the last of the repeated headers in the re-directed Invite.

Steps to Replicate: 

1. The SBC is configured with ERE, set the config required for handling embedded headers in 3xx as shown below.
   "set profiles signaling ipSignalingProfile DEFAULT_SIP egressIpAttributes redirect flags forceRequeryForRedirection enable honorEmbeddedHeadersIn3xx enable"
2. Run a UAS script with 302 redirection and add multiple "Identity headers"
   as shown below.
   Ex- Contact: 1234567890<sip:1234567890@10.xx.xx.xx:xxxx?Identity=test1&Identity=test2&Identity=test3>
3. Run a 2nd UAS Script to handle the Redirected INVITE.
4. Run SIPP call, trigger UAC script.
   
   Expected Result: The redirect INVITE shall contain both the Identity headers received in 3xx as embedded contact header.

The code is modified to support repeated SIP headers.

Workaround: None.

The customer SBC failover with SCM Process core due to Memory corruption.

Impact: When prack enable on ingress and advanced SMM for "variableScopeValue message" enable, the SBC may core when Egress responses 18x/2xx too fast trigger delay on ingress due to prack pending.

Root Cause: On ingress when 18x/2xx is queuing, it did not allocate proper resources for "variableScope" data. By the time it start to send out, the SBC access garbage data.

Steps to Replicate: This is random issue so may/may not reproducible.
In the Egress: inbound config variableScopeValue message and send to ingress.
In the Ingress: prack enable, outbound read the var1 and append to the userName of To Header.
Egress: Send multiple 18x/200 fast will trigger some delay on ingress for sending. Run load.

Properly allocated resource for "variableScope" data to address the issue.

Workaround: Disable prack.

The SBC reinvite with sendonly causing one-way audio.

Impact: The SBC unexpectedly send out reInvite sendonly when relay DPM is disabled.

Root Cause: There was an internal logical error to queue the sdp on ingress when egress send 18x (sendonly)

Steps to Replicate: 

1. Enable minimize and disable relay DPM on ingress.
2. A call B, B answer 183 (sendrecv), 183 (sendonly), 183 (sendrecv)...200 (sendrecv).

The code is modified so when subsequent 183 (sendonly) received on egress, the SBC updates the queue sdp (recvonly). Since the relay DPM is disable, the SBC should not change the direction of datapath on ingress.

Workaround: n/a. This is application bug per rfc, and the SDP should not change in subsequent 18x.

A SAM Process core dump was seen while executing OCSP stapling feature.

Impact: The SAM Process cores randomly when running a TLS call with the OCSP Stapling enabled.

Root Cause: Periodically, the SIPCM finds socketPtr to be NULL that is causing an invalid access of a NULL pointer.

Steps to Replicate: Running a TLS call with OCSP Stapling enabled.

The code is modified to check a NULL for the socketPtr.

Workaround: When the OCSP Stapling is disabled, the SAM process does not core.

Portfix SBX-100237: The CpxApp process is dumping the core on the managed node when callMediaStatus CLI is run from OAM node.

Impact: There was a CpxAppProc core when accessing the callMediaStatus of managed VMs from OAM through the node branch.

Root Cause: There was a data serialization bug that caused the issue.

Steps to Replicate: Access the callMediaStatus of managed VM from OAM through node branch.

The code is modified to address the core issue.

Workaround: None.

Portfix SBX-101407: The CpxApp process is dumping the core on the managed node when we get the NTP from OAM node.

Impact: The CpxProcess cores on an application shutdown.

Root Cause: There was an improper ConfdProxy object destruction.

Steps to Replicate: Stop the application while the traffic load is running is best to reproduce.

The code is modified to properly delete the ConfdProxy object.

Workaround: None.

Portfix SBX-104015: The RE-REGISTAR was shorter than 3600 secs due to the child AoR. There needs to be more detail about this behavior.

Impact: When the 200 OK response to REGISTER request contains a P-Associated-URI
header, the SBC creates a child AOR and forwards all REGISTER refresh requests to the registrar (effectively disabling the SBC fast refresh response).

Root Cause: The creation of child AORs cause all REGISTER refresh requests to be sent to the registrar (effectively disabling SBC fast refresh response).

Steps to Replicate: 

1. Configure the SBC to perform registration.
2. The 200 OK response to REGISTER request contains the P-ASSOCIATED-URI header.
3. In the error case, the refresh REGISTER requests are all forwarded to the registrar.

Removed the child AOR check from SipRaRegisterRequestCompletedAorNfy(), permitting the SBC to send a fast refresh responses.

Workaround: None.

Calculated the size of a buffer as too small and error logs were being generated post upgrade.

Impact: Removes this incorrect log message:

SipsRedGetCallControlSizeCmd()
Minor .SIPSG: *SipsRedMirrorCallControlCmd: Potential for buffer
format ERROR. Available:16076 Calculated:383 Packed:277

Root Cause: This message is logged incorrectly when the code is calculating the size for a Redundancy message.

Steps to Replicate: The steps cannot be replicated.

The code is modified to remove this incorrect log message:

SipsRedGetCallControlSizeCmd()
Minor .SIPSG: *SipsRedMirrorCallControlCmd: Potential for buffer
format ERROR. Available:16076 Calculated:383 Packed:277

Workaround: N/A

SBX-102629

2

PRS process memory leak for object based messages.

Impact: There was a slow memory leak in PRS.

Root Cause: Object based messages are not properly being deallocated. This leaked both fm fault notifications and fp stats request messages.

Steps to Replicate: Overtime watch PRS memory size increase.

The code is modified to properly free the messages.

Workaround: No workaround.

The SBC modifies the Replaces parameter with the wrong Call-ID and tags if the Replaced call was looped back to the SBC through a SIP Proxy that does not modify SIP Call-ID and tags.

Impact: Relay a refer with replaces for loopback call, the SBC picks up the wrong call leg.

Root Cause: The SBC query for the replaces callId, and found matching both legs (loopback).The SBC pick up the wrong leg.

Steps to Replicate: This is specific to customer call flow.

If loopback is detected, only pick the one with to-tag match local-tag. This is per RFC-3891, section 3.

Workaround: N/A

The Route data lost after the offline PM upgrade from 625R0 to 823A13.

Impact: The special character data was not getting migrated to postgres version.

Root Cause: The special character was causing issues with the Postgres data loading.

Steps to Replicate: 

1. Created 100+ routes that have special chars in its DN in 625R0.
2. Upgraded to 823A13 through the PM offline upgrade.
3. After an upgrade, all route data was lost while other data are unaltered.
4. System is up and services are running but no clue of route data.

The code is modified to escape the special characters.

Workaround: No workaround.

Portfix SBX-100910: A CE_2N_Comp_CcsP coredump is seen in SWE after disabling the interface.

Impact: A CCS Process coredump was observed due to a healthcheck failure. This occurred while the software was establishing a connection to the third-party software serf, which it starts up to detect and communicate with other nodes in the cluster.

Root Cause: The healthcheck failure occurs when a software object does not respond quickly enough to a health check message. This occurs because the software was waiting for serf to finish starting up and did not respond to the message in time.

Steps to Replicate: Stop/start the SBC with the networking down or disable/enable the interfaces on a running system. These steps will trigger the code to start up serf and reconnect to it.

The code is modified to introduce a new timer for when the serf is started up, to allow the software to remain responsive to other messages. As an added precaution, healthchecks are temporarily disabled while connecting to serf, to avoid any chance of a healthcheck failure.

Workaround: N/A

The code is modified to correct the position of a DSCP value while calling the SipCmSendIncomingPduUind in SipCmProcessRemotePdu.

Workaround: N/A

The code is modified for the Allow message body QSIG Content-Type: multipart/mixed, Resource/Lists across GW-GW.

Workaround: N/A

SBX-103098

2

The To tag in To header is corrupted when the SBC forwards the OOD Message.

Impact: The To Tag in OOD messages from the registered users are getting corrupted when relayed.

Root Cause: This is because of a logic issue in the stack function.

Steps to Replicate:

1. Register an End Point.
2. Send a MESSAGE request from same source IP/Port from registered endpoint.
3. Perform a switchover.
4. Send a MESSAGE request from same source IP/Port from registered endpoint.
5. Send a MESSAGE request from different source IP/Port.

The code is modified in stack function and also rejected the OOD messages from the Registered users with "To Tag", similar to SBX-76003 that was done as part of recent customer requirement.

Workaround: No workaround.

SBX-102992 | SBX-103001

The SBC is blacklisting a peer when the mid dialog request gets timed out even though "midDialogArsScreenLevel" flag is set to "never" in sipArsProfile.

Impact: The SBC is blacklisting peer when a mid dialog request gets timed out even though the midDialogArsScreenLevel flag is set to never in sipArsProfile.

Root Cause: The original design only covered original INVITE messages and not mid-call INVITE messages.

Steps to Replicate:

1. Make a call from UAC.
2. Answer the call on UAS.
3. When the call is stable send a Re-INVITE request from UAC.
4. The SBC forwards this Re-INVITE request to UAS.
5. Do not send any response to this Re-INVITE from UAS.
6. The SBC sends BYE and terminates the call.
7. Check the "sipArsStatus" on egress zone.

The code is modified to correctly handle mid-dialogue INVITE's as well as original INVITEs.

Workaround: None.

Portfix SBX-85825: The SBC Application fails with LCA errors on an initial EMS Cluster Configuration push.

Impact: There was a packet drop between the SBC and EMS while downloading the configuration.

Root Cause: Due to small fill rate and bucket size, traffic between the SBC and EMS is flow controlled.

Steps to Replicate: Reboot the SBC so that it registers with the EMS and download the configuration from EMS. Capture the packets between the two to ensure that there is no drop.

Increased the fill rate and bucket size to accommodate large configuration size (~40MB).

Workaround: N/A

Porfix SBX-102401: The encryptedStore confd_cmd GET operations taking too much time during switchover (Nto1).

Impact: The encryptedStore confd_cmd GET operations were taking too much time during a switchover (Nto1)

Root Cause: The confd_cmd GET calls were taking too much time to return during a switchover if the confd was under stress.

Steps to Replicate: 

1. Launch Nto1 set up (preferably with a big configuration).
2. Perform a switchover on active.
3. On standby (becoming active) verify (during switchover) that:
   
   1. The confd_cmd get cmds are not taking too long to return (ideally ~ 0.2 sec)
   2. The ChmProcessIntanceData does not take too long to finish.
   3. The ChmClearEncryptedStoreOfPeers returns immediately but encryptedStore.sh deletePeerEntries continues in background and finishes relatively faster (than before).
   4. EncryptedStores are consistent on standby and actives.

Use the confd_cmd -f option with an empty string argument
while running the encryptedstore script in background to save some time and address the issue.

Workaround: N/A

The warning message needs to be updated/have a proper ending when deleting the ipInterface.

Impact: A warning message does not display in the cloud SBC when deleting the IP interface. But there is no problem with functionality.

Root Cause: This is working fine in the hardware SBC. The problem is in the cloud SBC when the warning message is displayed, a few values are checked to see if they arr matching and then we show the warning popup. In the case of the cloud SBC, the values index are different and the check is failing because we are unable to show the warning popup when we delete the IP Interface.

Steps to Replicate: 

1. Login to EMA.
2. Go to All -> Address Context ->IP Interface Group -> IP Interface.
3. Click on New IP Interface button.
4. Fill the correct info in text field, select state as enable, Mode as In Service and click on save button.
5. Creation should be successful.
6. Now select the same entry from IP interface list section and click on delete button.
7. We should be able to see warning popup like "'addressContext PPP_AC ipInterfaceGroup PPP_IG ipInterface': Deleting ipInterface while in dryUp action can cause problems with calls on the ipInterface. Use force action to clear existing calls on the ipInterface before deleting. Do you wish to continue?"

The code is modified to handle the cloud SBC scenario as well.

Workaround: N/A

The SIPCM (SAM) deadlock reported on the SNMP request.

Impact: The SAM process cores when getTlsStatus command is executed.

Root Cause: The issue is because when fetching tls Status, Design was trying to access sessData that was no longer valid.

Steps to Replicate: Execute the getTlsStatus command when running TLS load. The SAM process cores randomly.

The code is modified to fetch the status from socketPtr instead of the sessData.

Workaround: N/A

Portfix SBX-98954: Unable to configure HW/SWE SBC's into an SLB deployment.

Impact: The SLB cannot be supported with the H/W SBCs.

Root Cause: The CPX validation check prevented configuration on the H/W SBCs to configure the SLB.

Steps to Replicate: 

1. Configured the SLB.
2. Configured HW/SWE SSBC in the order below:
   1. Enabled the SLB usage.
   2. Configured the SLB common Interface.
   3. Configured the 'slbAddress' and 'ipAddress'.
   4. Attempted configuring the zone and ssp.

The code is modified to allow configuration of SLB on the H/W SBCs.

Workaround: N/A

Portfix SBX-101887: The SBC was generating duplicate ICID for different calls.

Impact: The same ICID value was generated for two different calls.

Root Cause: The clock resolution used was 10ms leading to using the same timestamp when subsequent calls were received within 10ms.

Steps to Replicate: Run load with CDR enabled. Check no two calls have the same ICID.

Use high resolution time tick of a 1 micro second and add a counter for achieving granularity of 100ns.

Workaround: N/A

SBX-102444 | SBX-103405

2

Portfix SBX-102444: The link does not get restored for the mgmt interface for an I-SBC in centralized mode.

Impact: The link state in the portMonitorStatus command does not show the correct state when the LDGs are disabled and enabled in a specific order.

Root Cause: In a HA set up, each instance (for example, Node A and B) maintains its own Port Monitor configuration where as both instances maintains the link monitors configuration of both nodes.

When the state of Link Monitor configured on Node B is modified, this CDB notification goes to both nodes A and B. As both of them have the configuration, they update the state and reset a link status maintained in a port monitor to UNKNOWN.

Then, ICMP Pings are sent only on node B. When it receives a response, it updates the link state of a portMonitor configured on that node and generates a event notification.

When the node A gets this event notification, it recognizes that this notification is from peer and only updates the Link Monitor fault state and port monitor link state remains as UNKNOWN on this node.

Steps to Replicate:

1) Instantiate the I-SBC in a centralized mode.
2) Configure the LDG for MGMT interface.
3) Disable the LDG for vsbc2.
4) Disable the LDG for vsbc1.
5) Enable the LDG for vsbc2.
6) Enable LDG for vsbc1.

The code is modified to not initialize link state of port monitor on local node when admin state of peer node LM configuration is changed.

Workaround: The sequence of the LDG state modification has to be changed.

The MGT0 cannot reach the GW.

Impact: The mgt0 interface appears to be inactive. The user cannot ping the gateway from SBC-5400.
The ethtool -S mgt0 shows the tx_pause count rising rapidly.

Root Cause: This problem is specific to SBX-5400 fix.

The SBC-5400 has a backpressure mechanism for management ports so that when the receive buffer exceeds a certain level, Design sends pause packets out to the interface to ask the switch/router to pause transmission momentarily.

The receive buffer count is incremented by a hardware module and decremented by software.
The notification was sent from Network Processor to application about sRTP Rollover Counter (ROC) reset is specifying the wrong interface so software decrements the wrong receive buffer count.

This causes the receive buffer count to wrap around, resulting in a high value. This triggers the back-pressure mechanism and Design continuously send out pause packets.

Steps to Replicate: 

1. Set up a pass-thru SRTP call.
2. Run a long duration sRTP call (about 30 minutes) so that the Rollover Counter (ROC) in the sRTP packet becomes non-zero.
3. Modify the call so that the sRTP packet's SSRC changes.
4. Check the pause frames of mgt ports.

Set the correct interface information in the notification to address the issue.

Workaround: 

1. Run the attached check_mgt0.py.txt (rename it to check_mgt0.py) python script from the Linux command.
2. Recommendation is to add it as a cron job and run it every 20-30 minutes.
3. How to add cronjob and run it every 20 minutes:
   copy check_mgt0.py to /opt/sonus/sbx/scripts
   As root running in Linux:
   crontab -e
   */20 * * * * /opt/sonus/sbx/scripts/check_mgt0.py

The "tgrp" parameter was not passing transparently when theSIP in the core is enabled.

Impact: The SipCore calls are passing the wrong tgrp parameter value.

Root Cause: There was missing logic to support the SipCore.

Steps to Replicate:

1. Configure the SipCore feature.
   1. Both IPSP core and egress leg have "originating Trunk Group Options" set to "Include Tgrp with IP address"
2. Make a SipCore call.

The code is modified to support the SipCore feature.

Workaround: N/A

Portfix SBX-100250: The KVM Direct-Single SBC not uploading configuration to the EMS.

Impact: 1:1 on KVM is not creating config store directory.

Root Cause: hwSubType variable value for KVM is not consider by lca.py script.

Steps to Replicate: Deploy the 1:1 on KVM.

Consider hwType variable value instead of hwSubType. This will work for all virtual deployment types to address the issue.

Workaround: N/A

Portfix SBX-101235: The OAM does not output the NTPQ command.

Impact: The NTP server CLI command not applied to the OAM.

Root Cause: The NTP server CLI command logic was skipped for OAM due to blind logic change from SmaIsConfigurator() to SmaIsOAM().

Steps to Replicate: CLI:

Configure:
set system ntp serverAdmin 169.254.120.4 state disabled
commit
set system ntp serverAdmin fd00:10:6b50:4500::11
set system ntp serverAdmin fd00:10:6b50:4500::11 state enabled
commit
<<<App on OAM nodes gets restarted>>>
request system admin vsbcSystem saveAndActivate
<<<App on managed nodes gets restarted>>>

Run the OAM prompt:
ntpq -p [is successful]

Remove erroneous check for OAM node type to address the issue.

Workaround: None.

Portfix SBX-101853: Observing an issue with the Restore operation in 9.1.

Impact: Restored the configuration is not applied to the Standby OAM and managed nodes.

Root Cause: Configuration updater scripts do not detect the restored revision and do not reboot to apply it. This occurs when only two revision entries are present in /mnt/gfsvol1/config-versions.txt

Steps to Replicate: Ensure only one revision entry is present in /mnt/gfsvol1/config-versions.txt.
Restore to a previous revision.

The code is modified to check the revisions available.

Workaround: None.

Portfix SBX-102123: There was a CpxA process core on the OAM upon configuring the NTP server.

Impact: The CpxProcess cores on an application shutdown.

Root Cause: There was an improper ConfdProxy object destruction.

Steps to Replicate: Stop the application while the traffic load is running is best to reproduce.

The code is modified to properly delete the ConfdProxy object.

Workaround: None.

Portfix SBX-101791: Observed the CpxA Process coredump on the SBC with 10% Packet loss introduced on HA interface.

Impact: The CpxProcess cores while traffic load is running.

Root Cause: Segmentation fault in MsgClient::checkBuffering.

Steps to Replicate: Execute the traffic run.

The code is modified to address the issue.

Workaround: None.

Portfix SBX-101555: An application timeout error for "show table node" on OAM.

Impact: The 'show table node' aborts with a timeout.

Root Cause: There is no handling for empty statistics.

Steps to Replicate: Execute a 'show table node'.

The code is modified for the empty statistics by returning an empty response.

Workaround: None.

Portfix SBX-96018: Configuration and Profile Import/Export fails on the OAM SBC nodes.

Impact: The OAM and managed nodes have access to user-config-import CLI command. Configuration issues can be introduced if this command is used as OAM manages the configuration.

Root Cause: There is no display condition check for the user-config-import CLI command.

Steps to Replicate: 

Test #1:
- Deploy the VNF with OAM.
- Check that user-config-import CLI command is not available on OAM and the managed nodes.

Test #2:
- Deploy the VNF without OAM.
- Check that user-config-import CLI command is available on nodes.

The code is modified for the user-config-import CLI command.

Workaround: Do not use the user-config-import command.

Portfix SBX-100805: The LCA does not exit even if the DRBD configuration fails in a 1to1 mode.

Impact: The LCA does not exit even if the DRBD configuration fails in a 1to1 mode.

Root Cause: The reConfigureHa.pl (Which configures DRBD) script's execution status was not checking in the LCA.

Steps to Replicate: Removed the DRBD disk and try to bring up the SBC. With the changes, the LCA run stopped.

The code is modified to check the status of execution of the reconfigureHa.pl script and when it fails, stop running of LCA.

Workaround: None.

Portfix SBX-102146: Observing the "FAC: *FacSendGetRecordRequest: Get Record request already Sent:192.xxx.x.xxx" major log on an overnight load.

Impact: Getting the FacSendGetRecordRequest major log during the overnight SBC load run.

Root Cause: The FacSendGetRecordRequest Log is made as a major log but ideally it should be info level log.

Steps to Replicate: Run the Overnight SBC load and check for FacSendGetRecordRequest major log.

The code is modified to address the issue.

Workaround: None.

Portfix SBX-103798: There was dual NUMA support in SWe.

Impact: The SWe does not come up in VMs having multiple NUMA nodes except for signaling traffic profile (SLB, SSBC, SO-SBC).

Root Cause: The root cause was due to deliberate software restriction to not allow multiple NUMAs.

Steps to Replicate: 

1. Configure a KVM instance with dual NUMA and install in Non-Gold/Non-GPU set up.
2. Let it come in default traffic profile.
3. Verify that SBC would come up fine without any HostCheck errors.

The code is modified to allow multiple NUMA irrespective of personality and profile.

Workaround: No workaround except using single NUMA set up.

Portfix SBX-73003: A vulnerability in use of JavaScript Library To 8.2.4.

Impact: The vulnerability in use of the Javascript library.

Root Cause: When the Javascript library was using version 1.8.0., the vulnerability was reported.

Steps to Replicate: Verified the appropriate screens to test and it is success.

The code is modified to fix the vulnerability. To upgrade this Javascript library, upgrade the jquery-ui (1.12.1) and dataTables (1.10.20).

Workaround: N/A

A DTMF 2833 PT change (in offer) was incorrect the OA SBX handling (wrong answer PT).

Impact: When there is a peer offer with new PT, the SBC is unable to process and response the previous one.

Root Cause: There was a logical error that checked the wrong field to detect PT change.

Steps to Replicate: 

1. A call B and connect with "0 100".
2. A Invite with offer new PT "0 101".
3. The SBC responds with "0 100"

The code is modified to check the correct field for PT change.

Workaround: None.

The SBC sent unnecessary 481 for PRACK when the CANCEL is received.

Impact: The SBC sends unnecessary 481 for PRACK when receiving a CANCEL.

Root Cause: When the SBC received PRACK in response to a 18x message, it generated it was added to an outstanding transaction list. But it was not removed from the list when the SBC sent the subsequent 200 OK for the PRACK. When performing a CANCEL then it arrives, the SBC thought the PRACK was still an outstanding transaction and generated a 481 message.

Steps to Replicate: 

1. A call B, B response 180 and send 180 to A with PRACK required.
2. After 32 seconds A send a CANCEL, the SBC sends unnecessary 481 PRACK

The code is modified to remove the PRACK from the outstanding transaction list when the SBC sends back the associated 200 OK, so there is no extra 481 generated if CANCEL then arrives.

Workaround: Enable the e2e PRACK if the egress also support PRACK.

Portfix SBX-90884: Performance: Observed a SCM memory leak while running the STIR/SHAKEN with RPH Signing/Verification.

Impact: The memory leak is caused at the SCM process for a STIR-SHAKEN failure.

Root Cause: 

When a STIR-SHAKEN call is performed and the STIR-SHAKEN functionality resulted in failure, then the STIR-SHAKEN "Reason Code Text" is populated in order to know the reason for the failure.

As a result, the STIR-SHAKEN failure "Reason Code Text" is then communicated in backward messages.

While populating "Reason Code Text", the intended function will execute twice for a given call. For the second invocation, the existing "Reason Code Text" memory was neither reused or freed. Instead, the call was newly allocated each time for the same call.

This resulted in memory leak for every STIR-SHAKEN call failure.

Steps to Replicate: 

1. Run the configuration and execution of a STIR-SHAKEN Call.
2. The STIR-SHAKEN functionality to result in a failure.

The code is modified to fix the following:

1. If "Reason Code Text" memory is already allocated before, then the same memory is reused again by resetting the memory.
2. If "Reason Code Text" memory is not allocated previously, then new memory is allocated and used.

Workaround: None.

Portfix SBX-101727: The LeakSanitizer detected memory leaks in the SAM Process (DFe process).

Impact: When making configuration changes to the D-SBC profile, the SBC leaks a small amount of memory.

Root Cause: The SBC application code uses local buffers while reading profile changes from the CDB and this memory was not getting freed up correctly, leading to a leak.

Steps to Replicate: This problem is highlighted when using ASAN enabled build in the engineering lab and making DSBC profile changes.

The code is modified to correctly access the local buffers to avoid the memory leak.

Workaround: None.

Portfix SBX-102056: The AddressSanitizer detected a heap-buffer-overflow in SipsgRedundMsgParamLoadSubsCbStr.

Impact: While printing the debug messages as part of making subscriber data redundant, the SBC was reading off the end of a buffer.

Root Cause: Some of the buffers used to hold string information were not large enough to include a NULL terminator so when printing out debug messages, the printing code was reading off the end of the buffer.

Steps to Replicate: This problem is highlighted when using ASAN enabled images in the engineering lab.

The code is modified to ensure the strings for the subscriber data are NULL terminated to avoid the code reading of the end of the string buffer.

Workaround: None.

Portfix SBX-101597: The LeakSanitizer detected memory leaks in the CpxAppProcess.

Impact: When making changes to the D-SBC signaling port configuration, the SBC is leaking a small amount of memory.

Root Cause: While reading configuration changes from the CDB, the SBC application code allocates local buffers and they are not getting freed up at the end of processing leading to a small memory leak.

Steps to Replicate: The issue is seen when using ASAN enabled images in the engineering lab and making configuration changes to the DSBC signaling port configuration.

The code is modified to correctly free the memory allocated to the local buffers to avoid the memory leak.

Workaround: None.

Portfix SBX-103603: The LeakSanitizer: detected memory leaks in the MrfRmProcessTrmAllocRpyMsg.

Impact: While testing call scenarios for RTCP for T.140 Pass-through functionality, it was observed that the SCM process could leak memory for calls associated with an MRF (external media transcoder).

Root Cause: The SBC was allocated memory while processing the SDP associated with this call but was not always freeing up the memory at the end of the call.

Steps to Replicate: Run various call scenarios with MRF where the SBC is using the SIP to allocated media resources on an external media transcoder (MRF) or T-SBC.

The code is modified to correctly free all the memory allocated for the call.

Workaround: None.

Portfix SBX-103489: The LeakSanitizer: detected memory leaks at the confd_malloc.

Impact: The small memory leak during the configuration of lawful intercept server.

Root Cause: The configuration code was reading information from the CDB and storing information in an internal memory block but not freeing it up.

Steps to Replicate: Create a lawful intercept server and make configuration changes.

The code is modified to correctly release the internal memory block at the end of the configuration action.

Workaround: None.

Portfix SBX-102054: The LeakSanitizer detected an error in CpxAaaGetElem.

Impact: When adding new users, a small amount of memory leaked.

Root Cause: While processing the addition of new users, the code allocated temporary memory blocks to hold information retrieved from the CDB about existing users this memory was not getting completely released after use.

Steps to Replicate: This issue is observed when running with ASAN images in the lab.

The code is modified to ensure the temporary memory allocated is freed up after use.

Workaround: None.

Portfix SBX-102060: Leak detected while running the SBX-93279 HA cases, especially on the SCM and IM process.

Impact: When making a CAC profile configuration change, the SBC leaks a small amount of memory.

Root Cause: When reading configuration changes from CDB, the application code was storing information in local buffers and not freeing it up correctly, leading to a memory leak.

Steps to Replicate: This problem is highlighted when using ASAN enabled images in the engineering lab and making CAC profile configuration changes.

The code is modified to correctly free up the memory in local buffers to avoid the leak.

Workaround: None.

Portfix SBX-101504: The SPD for IPSec SA was created for the 3GPP IMS AKA should allow both the TCP and UDP transport protocols.

Impact: The Apple and Samsung phones run the IMS AKA registration over the TCP, but send an SMS (MESSAGE) over the UDP later, which was dropped on the SBC.

Root Cause: The SBC honors messages only on that transport for IPsec (established using the IMS AKA) that was used initially during registration.

Steps to Replicate: 

1. Set up SIP registration using IMS AKA on TCP.
2. Make SIP calls (INVITE) over TCP.
3. Exchange SMS (MESSAGE) over UDP.

The code is modified so the SBC now listens on UDP ports also, same ports that are exchanged during initial IMS AKA registration, along with the TCP ports.

Workaround: N/A

Portfix SBX-101749: The SBC incorrectly passed a complete contact header transparently without the presence of the isFocus parameter.

Impact: The SBC incorrectly passed a complete contact header transparently without the presence of the isFocus parameter.

Root Cause: The SBC passed a complete contact header transparently without the presence of the isFocus parameter, with the contactTransparencyForIsFocusMediaTag enabled on both trunk groups.

Steps to Replicate: 

1. Configure the SBC for A to B call.
2. Enable the flag contactTransparencyForIsFocusMediaTag on both TGs.
3. Make a SUBSCRIBE-NOTIFY call flow.
4. Ensure the isFocus parameter is present in the Notify message, but is not present in the 200 OK response.

The code is modified to not transparently pass the complete contact header when the contactTransparencyForIsFocusMediaTag flag is enabled, and the isFocus parameter is not present in the 200 OK notification.

Workaround: N/A

Portfix SBX-88376: The SBC interop with Ribbon AS: Upstream Forking was not working consistently using the TLS.

Impact: For a non UDP transports, in an upstream forking scenario where as on ingress side, the SBC receives multiple INVITE with the same Call-Id, From, and To.
In such a scenario, the SBC is not sending all the received INVITEs out to the end points that are registered on same username.

Root Cause: The SBC used to drop off those forked INVITE's that are received during the processing of initial INVITE.

Steps to Replicate: On a non UDP transport, send multiple forked INVITE's towards the SBC with the same Call-Id, From, To, Request URI.

Queuing the subsequent forked INVITE's and processing them in the same order received solves the problem.

Workaround: None or use the UDP as transport.

Portfix SBX-103775: Changing the XRM to build ALL media RID’s static IP headers with the “Don’t Fragment” (DF) bit SET TO 0 instead of 1.

Impact: The SBC forwards the DTLS packets correctly but the large certificated packet does not reach the calling device.

Root Cause: The SBC has set the DF (do not fragment) bit in IPv4 header so routers cannot split the packet and it can result in the packet getting dropped.

Steps to Replicate: 

1. Set up a Inter-work between Full ICE client and ICE Lite call flow with DTLS/SRTP relay.
2. Caller send out large DTLS packets.
3. Capture the packets, ensure the DF bit has not set (0x0000).

The code is modified to set the DF bit when the SBC builds the IP header for DTLS and STUN packets.

Workaround: N/A

Portfix SBX-103939: The NAPT timer was not armed properly in the MoH.

Impact: The NAPT timer did not started as the NRMA requested.

Root Cause: The RTP flow mode was xmt-only. Then, the NRMA requested flow change to enable the NAPT timer expiry because we have already learned source address. But the XRM did not start NAPT timer because it was a learning next request.

Steps to Replicate: Run NAPT regression tests to reproduce the issue.

The code is modified to start the NAPT timer for learning next if NRMA has set XRM_NAPT_MEDIA_NAPT_TIMER_ACT_ENAB.

Workaround: None.

The `imfile' in rsyslog.conf without freshStartTail="on" can cause the SBC switchover when enabled.

Impact: When certain logs (consoleLog, sftpLog, syslogLog, platformAuditLog, ntpLog) get enabled initially, high number of old logs can be sent to the remote syslog server. This can cause network issues which can cause the SBC to switchover.

Root Cause: If a log is enabled for the first time, the rsyslog will try to send the whole log to the remote syslog server at once.

Steps to Replicate: Fill up /var/log/syslog with messages. Enable syslogLog in the SBC application. Verify older messages are not sent to the remote system.

The code is modified so the rsyslog option only reads in latest logs when initially enabled.

Workaround: Update the input rules in /etc/rsyslog.conf:

e.g.
#input(type="imfile" File="/var/log/session/session.*" Tag="console" Severity="info" Facility="lpr" escapeLF="on" addMetadata="on")

to:

#input(type="imfile" File="/var/log/session/session.*" Tag="console" Severity="info" Facility="lpr" escapeLF="on" addMetadata="on" freshStartTail="on")

The upgradeSBX.properties makes the EMA to show upgrade in progress forever.

Impact: An upgrade was completed but the Platform Manager Install/Upgrade screen gets stuck on previous upgrade and can not perform new upgrade operation.

Root Cause: We are facing this issue because we had updated the SBC upgrade steps as part of SBX-56893 Jira in 6.1.0. Whenever we upgrade the SBC from 6.0.x to 8.2.x, we will see the issue because there are different steps in 6.0.x and 8.2.x that is causing the issue.

When we upgrade the SBC using 6.0.x code that has the old upgrade steps once we execute the pre install check step, the PM API updates the upgradeSBX.properties file with next step (INSTALL_DB), the SBC goes for reboot and platform manager code gets replace with upgraded version which is 8.2.x.

Now, the PM API reads upgradeSBX.properties file to read the steps value that is INSTALL_DB and match the check but in 8.2.x PM code. There is no INSTALL_DB check in PM code because of that, we are unable to update the upgradeSBX.properties file with next steps.

Once an upgrade gets completed and the PM code updates upgradeSBX.properties file with FINISHED as steps, we move the upgradeSBX.properties file into upgradeSBX.properties.1, which is not happening and Platform Manager UI is unable to mark as complete and getting stuck on install/upgrade screen with previous upgrade.

Steps to Replicate: 

1. Use the ISO/app to install SBC 6.0.0F6 on a HW box. Install the SBC as a standby SBC on a SBC5110.
2. Go to EMA Administration > System Administration > Software Install/Upgrade and start an offline upgrade. Leave the browser alone, do not click anywhere in the EMA GUI once the upgrade has started.

The code is modified to handle the INSTALL_DB step check to mark upgrade as complete from Platform Manager UI.

Workaround: To perform again upgrade operation, delete the upgradeSBX.properties file from the SBC.

There are unwanted critical logs in HW SBC for:

"CRITICAL.SMA: *SmaIsCustomGPUProfile: Failed to read /opt/sonus/conf/swe/capacityEstimates/.activeCallMixInput.json".

Impact: The critical level log gets written into the DBG event log each application startup on Hardware Systems.

Root Cause: The system attempts to open a file that does not exist on Hardware Platforms.

Steps to Replicate: Startup the SBC application on a Hardware SBC. Verify the DBG log.

Stop the system from attempting to open the file on the Hardware Systems to address the issue. 

Workaround: None.

The DBG logs in the 7.2.4R0 SBCs filling up with the T38 logs.

Impact: When the call trace is enabled, the T38 messages are logged in a .DBG file. However, even after a call trace is disabled, T38 log messages continue to appear in .DBG file.

Root Cause: A gevelobal variable to set t38 log messages was set for call traces but never reset.

Steps to Replicate: 

1. Enable call tracing and make a T38 fax call.
   See that .DBG file has T38 log messages.
2. Disable call tracing and make a T38 fax call.
   T38 log messages do not appear in .DBG file.

The code is modified so after the call tracing is disabled, T38 log messages will not appear in .DBG file.

Workaround: Use 'unhide debug' command to disable T38 log message after call tracing is no longer needed.
> request sbx drm debug command "dspdebugt38 state disable"

Portfix SBX-102234: The SubsystemAdmin filter affects calltrace (TRC) logging showed useless logs.

Impact: After creating then deleting an entry in the oam eventLog subsystemAdmin, the original behaviour is not restored. INFO level events for that subsystem are no longer logged, even if the oam eventLog typeAdmin filterLevel is info.

Root Cause: Deleting an entry in oam eventLog subsystemAdmin does not restore settings back to default.

Steps to Replicate: 

1. Create an entry for a subsystem in the oam eventLog subsystemAdmin table.
2. Delete the entry.

The code is modified to process to deleted the oam eventLog subsystemAdmin entry so that settings are put back to default values.

Workaround: None.

Portfix SBX-98024 The contact header is not transparently passed when the Ingress and Egress had different transport types.

Impact: Contact header is not passed transparently from Ingress, when egress side has different transport type, even when the IPSP flag 'passCompleteContactHeader' is enabled on both ingress/egress Trunk Groups.

Root Cause: In API SipSgCheckAndSetContactHeaderTrasparency(), irrespective of transparency control is enabled/disabled, if the egress Sig Zone is MS Teams,
then contact header was not transparently passed to egress.

Steps to Replicate: 

1. Configure, IPSP flag 'passCompleteContactHeader to enable.
2. Attach the IPSP to both ingress/egress TG's.
3. Set Ingress transport to TCP / Egress to TL.
4. Create pathCheck profile with hostName 'sip.pstnhub.microsoft.com' and attach to the Teams side.
5. Make a successful call.

The code is modified so regardless of MS Teams zone, if the transparency control flag is enabled then pass the contact header transparently to the egress.

Workaround: None.

Mismatch in the count of ipPeers on Routing page.

Impact: There was a mismatch in the Zone, and IpPeer counts on the UI.

Root Cause: The backend logic using the contained predicate that fetches all the related data.

Steps to Replicate: Go to the Route screen, select trunkGroup and verify the Zone, IpPeer counts.

The code is modified from contained to match, and is now only associated data being fetch based on the parent selected.

Workaround: None.

Portfix SBX-86293: PreInstall Check improvements for file permissions.

Impact: PreUpgrade checks failure due to permission issues on external directory.

Root Cause: Pre-checks script failed to run commands on peer due to permission issues.

Steps to Replicate: Perform upgrade to the fix version and verify that upgrade is successful.

The code is modified to ensure permissions/ownership are set properly before running through pre-checks/upgrade.

Workaround: Set the right ownership for external directory as:
chgrp -R upload /opt/sonus/external/

The Out of Memory caused an SBC switchover.

Impact: Memory leaks when 3xx relay and reject 3xx on egress IPSP

Root Cause: Contact Headers in 3xx memory did not free properly.

Steps to Replicate: 

1. Enable 3xx relay and reject 3xx, trigger the SBC sends 503 to ingress.
2. A call B, B response 3xx with contact headers.
3. Or Enable 3xx relay and configure ingress "do not send 3xx", trigger SBC sends 503.
   

Ensure the memory of contact headers in 3xx are properly free when config to reject 3xx.

Workaround: Customer may using SMM to change the status 3xx to 503 instead.

The digitCriteria numberLength operation parameter was not configured after the first commit.

Impact: The digitCriteria numberLength operation parameter not configured after the first commit.

Root Cause: The value of the digitCriteria numberLength was overridden to 0 by the dpcIndicator during a Db update at first commit, which caused the issue. At the same time, the dpcIndicator was updating the value due to missing validation.

Steps to Replicate: Now in the first commit value digitCriteria numberLength should pick the value.

The code is modified to update the dpcIndicator only if the criteria type is URI. As a result, the digitCriteria numberLength is set expected value properly and issue got fixed.

Workaround: Committing twice will set the digitCriteria numberLength value.

There was an SM coredump, that was in communication with openhpi.

Impact: The SM took a bit longer to read all four DSP card version's info on the SBC7000. This leads to an SM coredump.

Root Cause: The SM took a bit longer to read all four DSP card version's info on the SBC7000.

Steps to Replicate: Test on the SBC7000 with all four DSP cards.

The SM time out value is increased from 90 sec to 120 sec to address the issue.

Workaround: None.

Portfix SBX-97650: ERROR: The leakSanitizer detected memory leaks.

Impact: The leak sanitizer detected a memory leak for NRMA Session Desc as part of the leak sanitizer tool execution.

Root Cause: During the SDP to PSP conversion, the routine SipSgConvertSdpToPsp() is allocating memory for the NSD structure and not freeing the memory when the conversion fails.

Steps to Replicate: Run the leak sanitized tool to reproduce the issue.

Freed the memory allocated for the NSD structure in cases where the SipSgConvertSdpToPsp() fails.

Workaround: N/A

Portfix SBX-103704: The RTP sourceAddressFiltering is not working (call leg dependency).

Impact: The source address validation is not done if call flow does not involve NAPT learning. For calls that want source address validation but does not have NAPT enabled, the SBC would not validate the source address and end up forwarding the RTP packet to the other endpoint.

Root Cause: There is no way for application to inform Network Processor to validate source if NAPT learning is not enabled.

Steps to Replicate: Involves a CISCO MOH server but probably something else can be used.

The code is modified to validate source even when NAPT learning is not enabled to address the issue.

Workaround: Enable NAPT learning for the call.

Portfix SBX-100253: The SBC services are not coming up with an ASAN build.

Impact: On the SBC5100/5200, there was an ASAN flags error while processing the negative Boot Init Response received from the DSP.

Root Cause: While processing the Boot Init response, variables that are not defined for DSPs on SBC5100/5200 is being accessed. Since memory is not allocated to these variables, accessing them is invalid.

Steps to Replicate: None.

The code is modified where if it is SBC5100/5200 DSP, the mentioned variables are not accessed.

Workaround: None.

Portfix SBX-103255: Enhance the sbxPerf on the SBC for lesser resource consumption.

Impact: The SBC performance monitoring tools like top and top2 at times take 20% of a CPU core there by reducing total available CPU resources for management activities on the SBC.

Root Cause: The sbxPerf that contains list of tools such as top, mpstat, and iostat currently runs periodically without any linux value configured. This can cause these processes to get prioritized and scheduled ahead of other management processes such as ConfD and SSH.

Steps to Replicate: Install fix build and ensure processes such as top, top2, mpstat sbxPerf are running with corrects value of 15 using the top command.

The code is modified so that priority of these processes are less compared to other management processes on the SBC.

Workaround: None.

Portfix SBX-103593: The SBC is ignoring Use Max Bitrate Only flag.

Impact: The SBC is ignoring Use Max Bitrate Only flag.

Root Cause: During Re-Invite answer processing, NrmaAdjustPeerT38MaxBitRate() routine is doing T38MaxBitRate adjustment based on packet size that is changing T38MaxBitRate value from 14400 to 4800.

Steps to Replicate: Call Flow:

1. Do a basic call between UAC and UAS.
2. The UAS sends fax re-invite to UAC with T38MaxBitrate set to 14400.
3. The SBC sends Invite out with T38MaxBitrate set to 14400.
4. The UAC answers with T38MaxBitrate set to 14400 in 200OK.
5. The SBC sends out 200ok with T38MaxBitrate set to 4800.

Actual Result:

--UAS(Re-Invite) -----> SBC ----> UAC
T38MaxBitRate:14400 T38MaxBitRate:14400
--UAC(200ok for Re-Invite) ----> SBC ----> UAS
T38MaxBitRate:14400 T38MaxBitRate:4800

Expected Result:

--UAS(Re-Invite) ----> SBC ----> UAC
T38MaxBitRate:14400 T38MaxBitRate:14400
--UAC(200ok for Re-Invite) ----> SBC ----> UAS
T38MaxBitRate:14400 T38MaxBitRate:14400

The code is modified to relay the T38MaxBitRate in SDP, which is received from peer for Direct Media calls.

Workaround: None.

Portfix SBX-105416: The glusterfs is failing on the standby OAM after a reboot.

Impact: The Gluster server on a Standby OAM fails to start.

Root Cause: The Gluster heal check conditions are wrong.

Steps to Replicate: 

1. Reboot the Standby OAM node, wait 2 minutes, then reboot Active OAM node.
2. Both Active and Standby OAM come up successfully.

The code is modified to obtain the expected result.

Workaround: None.

Portfix SBX-91799: The segfault in pamValidator during PM login with incorrect credentials.

Impact: The segfault in pamValidator on a failed login for locked user.

Root Cause: The pamValidator defines a conversation function that is called by pam modules to exchange values. The pam module was freeing a struct member in the first call and accessing the same freed value in another call to the conversation function that was causing segmentation fault.

Steps to Replicate: Perform following steps on any of the SBC deployment and verify that segmentation fault does not happen:

## TEST 1: Ensure success for correct username and password:

1. Encode username and password to base 64 and set as environment variables USER and PSWD example:
   export USER="YWRtaW4=" ; export PSWD="U29udXNAMTIz"
2. Run pamValidator and verify it returns success.

##TEST 2: Authentication Failure for username and incorrect password:

1. Encode the correct username and incorrect password to base64 and export as env variables USER and PSWD.
2. Run pamValidator and verify it returns "Authentication Failure".
3. Run pamValidator again multiple times (at least 3 more) and verify the authentication failure and no segmentation fault.
4. Wait for 30 seconds and try TEST#1 with the correct password and verify it succeeds.

The code is modified so the pam_response struct properly in between calls to the conversation function.

Workaround: None.

Portfix SBX-103387: The Video NAT learning is failing on the D-SBC cloud.

Impact: The Video NAT learning failed in the D-SBC.

Root Cause: On the D-SBC, NAT learning for video stream was not processed successfully and as a result caused the video packets to be dropped by the SBC.

Steps to Replicate: 

1. Configure the SBC for an Audio and Video call.
2. Enable the Signaling NAT and Media NAT in the Ingress and Egress Trunk Groups.
3. Make an Audio and Video call between the NATed EndPoints EP1 and EP2.
4. After a call gets established, the EP1 and EP2 sends Audio and Video packets.
5. NAT learning happens for Audio and Video and then the packets are sent and received from EP1 and EP2.

The code is modified to process the NAT learning for the video stream in the D-SBC.

Workaround: None.

Portfix SBX-104560: The redirection NOA set wrong in the CDR.

Impact: The "Redirecting Orig Cd Num - NOA" subfield of "Redirection Feature Spec Data" is not set correctly in CDR record if the the Redirecting Original Called Number - NOA value is modified by PSX.

Root Cause: The existing code was not updating the value for the CDR record based on route specific information returned from the PSX.

Steps to Replicate: Test Set up
========

1. Test requires call routing through the PSX.
   Routing on PSX, set up to route incoming call through routing label with two routes:
   - route 1, through local egress trunk group 1 on SBC to UE2.
   - route 2, through local egress trunk group 2 on SBC to UE3.
2. On egress trunk group 1 only, associate a DM/PM rule to change the "Number Type" to International for "redirecting Original Called Number".
3. Set the crankback profile to enable attemptRecordGeneration and add reason 41 (503 is mapped to 41 in default SIP to CPC mapping profile)
   set profiles callRouting crankbackProfile default
       attemptRecordGeneration enabled reason 41

Test Procedure
============

1. Send INVITE to the SBC ingress, INVITE should include a diversion header e.g.
   Diversion: <sip:+4969667740185@xxx.xxx.xxx.xxx>;
   privacy=off;screen=no;reason=unknown;counter=1
2. Call is routed using route 1, respond from first egress with 503 Service Unavailable
3. Call is re-routed using route 2, respond from second egress with 503 Service Unavailable
4. Check the CDR records for call on the SBC that should have two attempt records -
   1. ATTEMPT1 - should have "Redirection Feature Spec Data" that has subfield 22 "Redirecting Orig Cd Num - NOA" as 3 (Unique International Number), because the number was updated by PSX to international.
   2. ATTEMPT2 - should have "Redirection Feature Spec Data" that has subfield 22 "Redirecting Orig Cd Num - NOA" as 2 (Unique National Number), because the number was not updated by PSX.

The code is modified to populate the CDR value from the route specific data returned from PSX.

Workaround: None.

The LeakSanitizer: detected memory leaks in DCmRestoreAllMetaVarsToStandbyContext.

Impact: A small memory leak was observed in the SAM process while performing a switchover from standby to active box.

Root Cause: The standby instance stores information about the metavars associated with signaling ports configured on the active instance. During the conversion from standby to active, the standby data is moved into active structures but the original standby memory blocks are not freed up correctly resulting in a memory leak.

Steps to Replicate: Perform a switchover from standby to active while running a basic call, no memory leak will be observed in the SAM process.

The code is modified to correctly free the memory associated with the standby data when it gets transitioned to active to avoid the memory leak.

Workaround: None.

Portfix SBX-103224: The ipsecSaStatus with local SPI is failing, as well as ikeSaStatus and ikeSaStatistics with ikeSaIndex is failing.

Impact: The show command for ikeSaStatus and ikeSaStatistics was not displaying information for the IPSEC calls of type ikev2 when trying to display with specific key field information.

Root Cause: The display logic was missing to output call information for the type ikev2 with specific parameters.

Steps to Replicate: Make IPSEC calls of type ikev2 and then run the status and statistics commands using the remoteSpi as a parameter.

The code is modified to correctly display information for the IPSEC calls of type ikev2 with specific parameters.

Workaround: The display prints out all information correctly when no parameters are specified.

Portfix SBX-98302: The LeakSanitizer detected memory leaks.

Impact: The LeakSanitizer detected a memory leak in the SIPSG while processing the REGISTER messages.

Root Cause: While processing a REGISTER message, the SBC performs a PSX query and gets back the primary domain information in the policy response. The SBC allocates memory to hold this information in the registration control block (RCB) but it was not getting freed up when the RCB was later released.

Steps to Replicate: Configure the SBC with Primary Registrar Domain info and run a registration scenario with ASAN enabled images.

The code is modified to free the memory associated with primary domain information to avoid the memory leak.

Workaround: N/A

Portfix SBX-103732: There was an error AddressSanitizer: heap-buffer-overflow on address

0x61d001429c18 at pc 0x556c876cf74c bp 0x7fb14f64fab0 sp 0x7fb14f64f260 READ

of size 3488 at 0x61d001429c18 thread T21 in the SAM Process.

Impact: In a DSBC environment while processing the remote media data command message the SAM process could read off the end of the received message. Reading off the end of the allocated message block could in the worst case result in a coredump.

Root Cause: The SAM process was assuming that the size of the received message was larger than it actually was and this resulted in reading off the end of the received message buffer. In most cases this does not cause a problem but in could potentially result in a coredump if the associated buffer is at the end of the addressable memory space.

Steps to Replicate: Run RFC4117 test case for audio transcoding and video relay.

The code is modified to not read off the end of the received memory block.

Workaround: None.

Portfix SBX-97838: Unable to create a sipSigPort on the cloud SSBC when the IP metaVar is previously used for a different sipSigPort.

Impact: Unable to create a sipSigPort on the cloud SSBC when the IP metaVar is previously used for a different sipSigPort.

Root Cause: The root cause for this issue is there was a check to skip any validations if the instance is cloud based in the addIpAddressToMetaTable and deleteIpAddressFromMetaTable functions. As a part of SBX-59421 feature, in 8.1 the check was removed in the addIpAddressToMetaTable and as a result duplicate entries were not allowed to be configured. However, it was retained in deleteIpAddressFromMetaTable function. Even after sipSigPort is deleted, stale entries would still remain and the sipSigPort creation was failing with same metaVar as a result.

Steps to Replicate: The steps cannot be reproduced.

1. Create a sipSigPort (with index 1) with ipVarV4 IF2.IPV4
2. Delete the sipSigPort
3. Create a sipSigPort (with different index 2) with same ipVarV4 as in step 1.

Error seen: addressContext default zone ZONE_IAD sipSigPort 2': IP address / port number must be unique within an address context

The check that was retained in deleteIpAddressFromMetaTable is removed.
Any further sipSigPort deletions will remove the entries from the metaTable as well.

Workaround: If the customer is not using 9.0 and still in older releases, the following procedure can be used to remove the stale entries:

1. Perform an unhide debug
2. Navigate to the configuration mode.
3. Execute this command to remove the IP entries whose corresponding sipSigPorts have been deleted:
   #delete addressContext default sipSigPortAddrMeta <ipAddr> <port>

Portfix SBX-97832: Unable to delete the leaf node like ipVarV4 or ipVarV6 in the sipSigport configuration in the I-SBC.

Impact: Unable to delete leaf node like ipVarV4 or ipVarV6 in sipSigport configuration as shown in the example below:

admin@isbcSwe-192.168.2.21% delete addressContext default zone SBX_89017_KL_1 sipSigPort 5 ipVarV6
[ok][2020-03-09 06:04:15]

[edit]
admin@isbcSwe-192.168.2.21% commit
Aborted: 'addressContext default zone SBX_89017_KL_1 sipSigPort 5 ipVarV6': There is no value for parameter - ipVarV6
[error][2020-03-09 06:04:16]

Root Cause: In the SBC systems that were not using the metavar configuration, the deletion of ipAddress was allowed. This was due to a bug in the code that wrongly checked for the metavar value before allowing the IP address deletion.

Steps to Replicate: Configure and delete the leaf nodes of the sipSigPort.

The code is modified so the deletion is allowed.

Workaround: N/A

Portfix SBX-96436: The SBC was adding a reason header even when the verification is successful.

Impact: The SBC should only add a reason header and text when the verification service replied with a failure response.

Root Cause: The SBC was adding the reason header when it is received from the PSX without checking whether the verification is SUCCESS or not.

Steps to Replicate: 

1. Configure the verification server to send a SUCCESSFUL response for STIR/SHAKEN.
2. When the SBC receives verification service from PSX, then the SBC checks for verification status and adds the reason header in provisional/final response.

The code is modified so the SBC checks failure status then only adds the reason header if it is available.

Workaround: N/A

Portfix SBX-102827: The SBC 5210 experienced an upgrade failure during Starting DB_RESTORE stage.

Impact: Failure during upgrade while creating database referential keys.

Root Cause: Creation of foreign key fails on table packet_service_profile (Child Table) column CODEC_ENTRY_FK9, as it had "0" in one of its rows and dbimpl.codec_entry table (parent table) does not have this value.

Steps to Replicate: Upgrade a dump that has "0" in the following. It should be successful.

packet_service_profile.CODEC_ENTRY_FK9 

The code is modified to check the data and nullify any value that is not present in parent table before adding the referential keys,

Workaround: None.

Portfix SBX-104216: The SBC relays STUN packets in RTP streams on pass-thru calls (no transcoding); on RTP to SRTP calls, the relayed STUN packet causes the ROC to reset back to 0 and the remote peer discards the encrypted media packets.

Impact: Sending Non_RTP packets mixed/relayed in SRTP stream from the SBC resulted in one-way audio at the endpoints in longer duration calls.

Root Cause: These non-RTP packets relayed eg. STUN packet causes resetting of encryption ROC back to 0 on the SRTP stream sent out. This issue results in the remote peer discarding the encrypted media packet in long duration calls from the SBC, where ROC is expected to be non-zero.

Steps to Replicate: Run a SRTP passthrough call with media for more than 15 mins, with 10 ptime. Then send a non-RTP packet (STUN, UDP TL t.38) in that stream for pass-thru relay, and observe the media in that call.

The code is modified to not mix Non-RTP packets with SRTP encryption/decryption processing. As a result, SRTP media stream ROC does not reset with these mixed packets, and the endpoint will not have media issues in longer duration calls.

Workaround: If longer duration calls are expected to relay STUN messages as well, disabling the SRTP will solve media issue. However, if SRTP must be enabled for security reasons, there is no workaround for this.

Portfix SBX-103788: ASAN ERROR: LeakSanitizer: detected memory leaks in the CpxAppProcess.

Impact: There was a small memory leak when making the configuration changes to the GW signaling port.

Root Cause: The configuration logic was reading the interface group name from CDB into a temporary local variable in order to perform validation logic but never freed up this memory at the end of the validation.

Steps to Replicate: This issue is highlighted in the engineering lab when performing GW sig port configuration changes on an ASAN enabled build.

The code is modified to correctly free the temporary memory to avoid the leak.

Workaround: None.

CE_2N_Comp_ScmProcess_2==4455==ERROR: LeakSanitizer: detected memory leaks at confd_malloc.

Impact: ASAN detected memory leaks while processing a E164 profile.

Root Cause: The memory was released while processing a E164Profile at the end of the configuration action.

Steps to Replicate: Create and modify the E164Profile configuration.

The code is modified to release the internal memory block at the end of the configuration action.

Workaround: None.

Portfix SBX-96307: The LeakSanitizer detected memory leaks at SipSgEmergencyProfileAddPrefix.

Impact: Observed a memory leak while adding and deleting the prefix/urnPrefix entries in an emergency profile.

Root Cause: On the deletion of prefix/urnPrefix entries, the memory not getting freed for the prefix list and this was causing the memory leak.

Steps to Replicate: 

1. Create an emergency profile.
2. Add the prefix entries.
3. Delete the prefix entries (repeat step 2 and 3).

The code is modified to free the memory whenever the prefix entries is deleted from the emergency profile.

Workaround: To avoid the memory leak, delete and recreate the entire emergency profile rather than just deleting only the prefix entries from the profile.

Portfix SBX-103731: The AddressSanitizer: heap-buffer-overflow on address 0x61a0000cb9d9 observed in the SAM Process while running OCSP feature.

Impact: When the OCSP stapling feature is enabled on the SBC and the code was processing, the response it writes to unallocated memory and in the worst case this could result in process coredumps.

Root Cause: While processing a OCSP response, the code was allocating a memory buffer large enough to hold the response, but then incorrectly writing one byte off the end of the memory buffer while attempt to try and null terminate the string.

Steps to Replicate: 

Set up - UAC > Dut<>Adapter -> UAS

1. Create OCSP profile by configuring the defaultResponder and stapling enabled.
2. Attach the ocsp profile to the TLS profile configured.
3. From Endpoint A, initiate the TLS call with Client Hello from the SIPP UAC having OCSP parameter in it.
4. The SBC should send server hello, certificate to the user client.

The code is modified to correctly terminate the OCSP response string without writing off the end of the memory buffer allocated to hold the response.

Workaround: None.

Portfix SBX-97904: The AddressSanitizer detected a stack-buffer-overflow on the address 0x7f994bb18d30 at pc 0x56314564c886 bp 0x7f994bb18c10 sp 0x7f994bb183c0 WRITE of size 16 at 0x7f994bb18d30 thread T7.

Impact: A stack-Buffer-Address overflowed for a V6 Address.

Root Cause: As part "SBX-66765:Support for IPsec for Media with UDP transport for LI" feature,
when LI server is configured with V6 Address, the stack-buffer was overflowing and as a result some unidentified behavior can be observed in the feature SBX-66765.

Steps to Replicate: This problem is highlighted when testing the following scenario in the lab with ASAN enabled images.

1. The SBC is configured with V6 Address for IMS LI UDP media.
2. Make a call.
3. The media Interception should happen.
4. The IPsec connection should get established for the SBC and peer.

The code is modified to handle V6 Address overflow.

Workaround: N/A

Portfix SBX-103801: ASAN: Observed a run time error in the M-SBC "runtime error: load of value 42, which is not a valid value for type 'bool'"

Impact: The M-SBC could potentially configure the wrong data path mode for a call.

Root Cause: No issues were observed while running this functionality on a regular deployment image. But while testing with ASAN it highlighted that some of the fields used in a call resource allocation message were not always initialized correctly. This can potentially lead to unexpected behavior e.g. SYS_ERRs, wrong datapath mode set up.

Steps to Replicate: This issue was highlighted while while running test suite related to RFC-4117 MRF Mid-Call modification Enhancement.

The code is modified to correctly initialize the resource allocation request fields to avoid issues.

Workaround: None.

Portfix SBX-93898: The "request sbx xrm debug command sec -stat gcid <gcid>" was not showing the enc and dec details on the SBC SWe.

Impact: This debug command in unhide section is not showing all the required fields populated in the SBC SWe.

Root Cause: The NP response is not framed in expected order to application layer.

Steps to Replicate: Run a SRTP call and the issue this debug command. The debug command does not show all the populated fields.

The code is modified so the NP Response is correctly framed in expected order to application layer.

Workaround: This debug command is to see SSN field value with RoC, all other details can be seen from show call mediastatus.

Portfix SBX-96116: The LeakSanitizer detected memory leaks at SipSgCopyRemoteAddressInfo.

Impact: Observed a memory leak during an early call termination.

Root Cause: Whenever a call terminated early at the ingress leg, some resources were not freed and it was causing the memory leak.

Steps to Replicate: Configure the SBC to reject an HPC call based on GETS-Feature Control Match and RPH header processing (For early termination).

The code is modified to free the memory during an early call termination.

Workaround: N/A

The SBC silently drops the second Notify when running the off board query for the eventDialog.

Impact: SBCs silently drop the second Notify within a Subscribe if it received simultaneously.

Root Cause: Currently, the SBC does not support multiple Notify when it requires an off board query for a dialogEvent of Notify.

Steps to Replicate: After a Subscribe, the Peer server sends multiple Notify (almost at the same time) with dialogEvent for an off board query.

The code is modified so the SBCs respond with 503 with retry-after for this case.

Workaround: None.

There was a PRS Process coredump on an active/A node in the middle of an upgrade.

Impact: The PRS Process cored during the LSWU while syncing ICE call data.

Root Cause: The ICE redundancy code is taking a long time to sync the call data with the standby during LSWU, causing a Healthcheck timeout and a core dump.

Steps to Replicate: The issue cannot be reproduced easily. 

The code is modified so the ICE syncs the call data to standby and addresses the issue. 

Workaround: None. 

After the switchover, a large amount of the DBG messages generated.

Impact: When TG block direction setting is modified before a switchover, modified block direction setting does not reflect on SIP calls/message relays after switchover.

Root Cause: The SBC does not modify block direction correctly on the standby node.

Steps to Replicate: 

1. Blocked the TG on SBC-a(ACT)
   SUBSCRIBE is blocked with error : 503.
2. Switchover from SBC-a to SBC-b.
3. Confirmed the status which TG was blocked from SBC-b(ACT).
4. Switchover from SBC-b to SBC-a.
5. Confirmed the status which TG was blocked from SBC-a(ACT).
6. Unblocked TG on SBC-a(ACT).
7. Switchover from SBC-a to SBC-b.
8. Confirmed the TG status from SBC-b(ACT), it displayed unblocked, when SUBSCRIBE is tried, the SBC sends 503 back to ingress.

With a fix, at step 8, the SBC will relay SUBSCRIBE message correctly.

The code is modified to ensure the block direction configuration works correctly after a switchover.

Workaround: 

1. Apply following workaround to toggle block direction

set addressContext AC zone zone1 sipTrunkGroup SIPTG1 blockDirection incoming
commit
set addressContext AC zone zone1 sipTrunkGroup SIPTG1 blockDirection non
commit

Resource memory congestion level 3 is approaching threshold 90 sample 81.

Impact: The SAM Process is leaking memory when the AKA is being used.

Root Cause: The SAM Process is leaking an AKA related structure when the code is handling an error case scenario.

Steps to Replicate: Unable to reproduce this issue. The root cause was found by code inspection.

The code is modified to correctly free the AKA structure in all scenarios.

Workaround: None.

Portfix SBX-101668: The LeakSanitizer detects the memory leaks at the hdb_handle_create.

Impact: The leak sanitizer was reporting some memory leaks because of memory allocation done in the CcSetLiCdcMediaTypeFromDb() function in a ccCsv.c file.

Root Cause: The CcSetLiCdcMediaTypeFromDb() function in memory is dynamically allocated for the mediaIpInterfaceGroupName but not freed, which caused the memory leak.

Steps to Replicate: Run a 3xx call with LI configuration on an ASAN build and this leak should not appear.

The code is modified to free the memory allocated to mediaIpInterfaceGroupName after the use of this variable is completed.

Workaround: None.

Portfix SBX-102151: D-SBC is not sending the rtp packets to other end after transfer in a transcoded call.

Impact: On the D-SBC, one way media was observed with no media towards calling party A post call transfer by B to C for the scenario: First call (A-SBC-AS, AS-SBC-B), Second call (B'-SBC-AS, AS-SBC-C), where AS=Broadsoft application server.

From packet capture on A-SBC leg of the single DSP transcode call, the sequence number of packets sent by the SBC was reset to zero after A is put off hold. This issue is not seen on the I-SBC.

Root Cause: In the D-SBC, with every modify (ex: hold/resume) a new DSP(s) is allocated for a transcode call. The RTP sequence number and RTP timestamp from previous the DSP deactivation were not being passed to the subsequent DSPs allocated in the context of the same RTP stream (SSRC).

Steps to Replicate: Ways to replicate the issue:

1. A simple UAC (G711)-SBC-UAS (G729) transcode call on D-SBC platform which uses a single DSP.
Perform a call hold and resume from UAS and observe the RTP packets sequence numbers towards UAC.
The sequence number is reset to zero post resume leading to one way media (no media towards UAC post resume).
2. With call transfer:

Test Set up:
A1,A2,A3 -------------DSBC--------------NS,AS,MS (HOSTED)

Preconditions:
1. Endpoints A1, A2, and A3 are registered with operator AS through SBC5K (DSBC).
Procedure:
2. A1 calls A2, answer the call on A2.
3. User A2 initiates a attended transfer to User A3, User A3 answers the call.
4. Disconnect the call from A1/A3.
5. After A1 is put off hold after successful transfer, the sequence number would be reset to zero leading to one way media (no media towards A1).

The code is modified to ensure the RTP sequence number and the RTP timestamp statistics of both G711 side of DSP and non-G711 side of previous DSP deactivation are passed to the subsequent DSP, so that the sequence number continues incrementing sequentially within the same media stream/SSRC instead of restarting from zero for every modify.

Workaround: No workaround.

Portfix SBX-104156: The RTT-TTY support verify call flow in the SBC.

Impact: Lower case characters from the t140 packet were not getting converted correctly to Baudot (tty). Also, some other characters were not getting converted as per V.18 A2 table.

Root Cause: The translation table from T140 to Baudot (TTY) only accounted for subset of characters that were present in Baudot (TTY) character set.

Steps to Replicate: 

1. Set up a T.140=>Baudot (AMRNB=>G711) call.
2. Create a T.140 pcap file with all UTF8 characters (128) and validate baudot generation.
3. Create a T.140 pcap file with multi-byte sequence and validate baudot.

The code is modified for:

• The t140=>Baudot translation table as per V.18 Annx2.
• Handling valid multi-byte (2,3 and 4 bytes).
• Handling of Byte Order Mark (BOM).
• Invalid byte sequence and substitute valid but unknown multi-byte seq with an apostrophe.

Workaround: Instead of lower case letters, use upper case letters.

Portfix SBX-98884: The LeakSanitizer detected memory leaks at the IcmAlloc2.

Impact: The ASAN shows a memory leak when run with the surrogate registration.

Root Cause: A memory leak caused by the ICM request message since it did not free the SURROGATE task.

Steps to Replicate: 

1. Install the ASAN build.
2. Configure the surrogate registration.
3. Once the registration complete, disable the surrogate conf and cleanup.
4. Verify the ASAN leak report.

The code is modified so now the ICM memory is freed in the SURROGATE task when receiving a request from the SIPFE.

Workaround: N/A

Observing the major log "XrmAsyncCmdErrHdlr: ERROR NpMediaIntf cmd 2a gcid" in all nodes of the 2:1 cluster.

Impact: Observed the major log "XrmAsyncCmdErrHdlr: ERROR NpMediaIntf cmd 2a gcid" in all nodes of 2:1 cluster.

Root Cause: As the error did not have a valid GCID, the error log becomes unidentified, since there is no action taken by application based on this error message.

Steps to Replicate: Set up:

1. An SBC 2:1 cluster with flavor 32vCPU/32GB RAM/100GB disk /2GPU per instance.
2. OAM 1:1 . Flavor 4vCPU/6GB RAM/25GB disk.
3. C3 - MGC.
4. Pktart.
5. EMS.
6. AMRWB-G711U call flow.
7. Load sharing enabled in C3 to distribute the load between two active nodes.
8. Custom traffic profile and codec profile are configured.

Procedure:

1. Ensure the SBC1 and SBC3 are active nodes and SBC2 is a standby.
2. Place calls at 394 CPS and 62s as CHT.
3. Calls are distributed between SBC1 and SBC3.
4. The below log messages are found in all nodes of the 2:1 cluster.

The code is modified to move the error message below the GCID validity check.

During the validation if any issue is found, there is only valid points.

Workaround: None.

Portfix SBX-103154: The SBC:9.1 Comp_EnmProcessMain experienced a core dump.

Impact: Observed an EnmProcess core dump in a AWS instance when the license is applied immediately after the SBC comes up.

Root Cause: When the SBC comes up, the EMA loads default configurations on to the SBC (in this particular case it was loading the trap target configuration) and at the same time, the automation suite run by Design triggered the license installation. The race condition between the EMA trigger and the license command caused a deadlock because no command was able to proceed to completion and EnmProcess cored.

Steps to Replicate: To simulate the simultaneous trigger from EMA (REST API) and CLI, the following script was running to create and delete the trap target. While the script was running, a CLI session was opened and license installation command was run:

#!/bin/bash
while : ; do
curl -kisu admin:admin -XPUT -H "Content-Type: application/vnd.yang.data+xml" https://<EMA IP>/api/config/oam/snmp/trapTarget/target1 --data '
<trapTarget xmlns="http://sonusnet.com/ns/mibs/SONUS-ORCA/1.0">
<name>target1</name>
<ipAddress>127.0.0.1</ipAddress>
<port>8163</port>
<trapType>v2</trapType>
<targetUsername>admin</targetUsername>
<targetSecurityLevel>noAuthNoPriv</targetSecurityLevel>
<state>enabled</state>
</trapTarget>
'
curl -kisu admin:admin -XDELETE https://<EMA IP>/api/config/oam/snmp/trapTarget/target1
sleep .5;
done

The code is modified to handle license installation to a thread so that it is not blocked by any other command and it does not block any command.

Workaround: Delay the license installation for a few seconds after the SBC is up and running.

Portfix SBX-100938: An SCM process core dump was observed on the SBC for Blind Transfer no Answer from the customer when replied with 603 Decline.

Impact: On the refer failure tone was not aborted. Due to this, the media resources were in the wrong states. So when the SBC tried to revert to original call, it crashed.

Root Cause: Design was not aborting the tone on refer failure. This lead to issue in media handling.

Steps to Replicate: 1. Initiate a basic C2C from the customer app through KL user to Cisco EP1.
2. Call is answered.
3. Initiate a transfer call from the customer app to PSTN POLYCOM EP.
4. After REFER for Transfer is received on the SBC, KL will be out of call.
POLYCOM EP do not answer the call and sends 603 Decline.

At this point, a crash should not happen and the SBC reverts to the original call.

The code is modified to abort tone on refer failure.

Workaround: None.

Portfix SBX-100515: Calls are getting rejected with 403 instead of 480, when configured subscribeBurstMax is more than subscribeRateMax.

Impact: The subscriber request are getting rejected with 403 instead of 480, when configured, the subscribeBurstMax is more than subscribeRateMax.

Root Cause: The EndPoint Rate policy for the SUBSCRIBER request putting into OOD (OUT OF DIALOG) request bucket and request handling was treated as SUBSCRIBER, as a result it response causes the code mapping was not working for subscriber.

Steps to Replicate:

1. Configure the subsIngressEndPointRatePolicing in internalSipCauseMapProfile. [set profiles signaling sipCauseCodeMapping internalSipCauseMapProfile <profile_name> causeMap subsIngEpRatePolicing sipCause 480].
2. Configure the internalCauseString ex: "subscription rate exceeded" for subsIngEpRatePolicing [set profiles signaling sipCauseCodeMapping internalSipCauseMapProfile <profile_name> causeMap subsIngEpRatePolicing internalCauseString subscription rate exceeded].

3. Set TG level subscribe rate as 4 in ingress Cac [set addressContext default zone <Ingress_zone> sipTrunkGroup <Ingress_TG> cac ingress subscribeRateMax 4 subscribeBurstMax 6].
4. Send 8 subscribes per second to SBC from ingress Trunk group.

The code is modified to keep the EndPoint Rate policy for SUBSCRIBER in the SUBSCRIBER rate policy bucket.

Workaround: None.

Resource memory congestion level 3 is approaching threshold 90 sample 82.

Impact: The SIPCM is leaking a structure that is associated with the SIPMM functionality.

Root Cause: The code to free this memory under certain error/edge scenarios is missing.

Steps to Replicate: Since we do not know the exact call flow that caused the customer to hit the error condition, we cannot suggest Test Steps.

The code is modified to free the SIPMM related structure in error scenarios.

Workaround: Since we do not know the exact call flow that caused the customer to hit the error condition, we cannot suggest a workaround.

Portfix SBX-103634: The LeakSanitizer: detected memory leaks in DCmRestoreAllMetaVarsToStandbyContext.

Impact: A small Memory leak was observed in the SAM process while doing a switchover from standby to active box.

Root Cause: The standby instance stores information about the metavars associated with signaling ports configured on the active instance. During the conversion from standby to active. The standby data is moved into active structures but the original standby memory blocks are not freed up correctly resulting in a memory leak.

Steps to Replicate: Perform a switchover from standby to active while running a basic call, no memory leak will observed in the SAM process.

The code is modified to correctly free the memory associated with the standby data when it is transitioned to active to avoid the memory leak.

Workaround: None.

Portfix SBX-103988: The ICE not working after an upgrade.

Impact: When a call with ICE changes from direct media to pass through, ICE learning does not get enabled on the call.

Root Cause: The code was not re-enabling the ICE FSM to start ICE learning when changing from direct media to pass through.

Steps to Replicate: 

Test 1
--------

1. Establish a call with ICE on ingress and xdmi on egress.
2. Send re-invite from egress without xdmi and complete the re-invite related signaling.
3. Send stuns to ingress media to complete ICE learning and verify call state changes to established and media can be exchanged between ingress and egress.

The code is modified to re-enable the ICE FSM when changing from direct media to pass through to allow the receipt and processing of stun messages to complete ICE learning.

Workaround: N/A

The PRS process cored in NP media interface function.

Impact: The PRS process cored in NP media interface function.

Root Cause: When examining the source code, we found the command sent to NP was not initialized properly.

Steps to Replicate: This cannot be easily reproduced.

The code is modified to initialize the command correctly.

Workaround: None.

Portfix SBX-98843: Problem with management of the maxptime and ptime in Direct Media mode.

Impact: Problem with management of the maxptime and ptime in Direct Media mode.

Root Cause: Generally, the SBC ignores the ptime value if only the maxptime attribute is present in incoming SDP. If "sendPtimeInSdp" IPSP flag is enabled and peer is sending both ptime and maxptime, the SBC is preferring maxptime value while sending ptime.

The existing IPSP flag "sendPtimeInSdp" makes the SBC send out a=ptime, irrespective of what we receive from the peer.

Steps to Replicate: 

R-i=20 R-e=20
-----INVITE--------> SBC -----INVITE-------->
ptime=20
maxptime=40 ptime=20

1. Do a basic call from A to B.
2. A sends ptime=20 and maxptime=40 towards the SBC.
3. The Invite going out from the SBC should contain ptime=20 as we enabled both "sendPtimeInSdp" IPSP flag and "preferPtimeInSdp" TG flag.

The code is modified when this new flag "preferPtimeInSdp" is enabled, the SBC prefers ptime value received in a=ptime over a=maxptime in an incoming SDP.

This new flag "preferPtimeInSdp" needs be enabled in conjunction with existing flag "sendPtimeInSdp". If the SBC is configured to send a=ptime (isendPtimeInSdp is enabled on
outgoing trunk), then the preferPtimeInSdp is enabled on incoming trunk and vice versa.

Workaround: None.

Portfix SBX-97818: The audio stream with port=0 appears in callDetailStatus.

Impact: The audio stream with a port=0 shows up in the .

Root Cause: Currently, for an audio stream, there is no condition for stream absent. As a result, an audio stream is retaining even if a port is 0.

Steps to Replicate: Video and Audio (Audio Port = 0)
1. Make a basic video only call from A-B.
2. On a re-invite, add a Audio stream with valid port.
3. On the final re-invite change Audio port to 0.
 
Existing Behavior:
When Audio port is 0, audio stream is displaying in callMediaStatus and callDetailStatus

New Behavior:
When the Audio port is 0 complete Audio stream is removed from callMediaStatus and callDetailStatus.

The code is modified for an audio stream so that in a Video and Audio (audio port = 0) call, the audio stream gets removed from callMediaStatus and callDetailStatus.

Workaround: None.

Portfix SBX-92770: The SBC is not relaying the INFO coming with the message body to the other leg.

Impact: After a call transfer, an in-dialog SIP INFO messages were not relayed by the SBC.

Root Cause: The code was missing to handle the relay of the SIP INFO messages received on a transferred leg.

Steps to Replicate: 

1. Establish an A--->SBC--->B call.
2. A transfers a call to C. The call is now between B and C.
3. Either Party B or C sends an in dialog SIP INFO message.

The code is modified to handle the relay of INFO messages received after call transfer.

Workaround: N/A

Portfix SBX-104670: Add the configStore information to the sysdump.

Impact: Missing the configStore data to properly investigate the configuration revision issues.

Root Cause: Missing the configStore directory content list in the configStore tarball produced by the sysDump.pl.

Steps to Replicate: Execute the sysDump.pl and check that virtualLogs/configStore tarbal contains the configStore directory content list.

Add a configStore directory content list in the configStore tarball produced by sysDump.pl to address the issue.

Workaround: Get the configStore directory content list directly from the node(s).

An incorrect "Egress Zone Name" in ATTEMPT CDR for a GW-GW call.

Impact: For a call with multiple routes that include one or more local routes followed by routes on another gateway (for GW-GW) scenario, if the local routes fail and then other gateway route is selected, the resulting ingress gateway CDR for the call has an incorrect egress zone name and ID.

Root Cause: When there are multiple routes, when a route fails the call cranks back to use the next route but when selecting the next route the code is not clearing the internal CDR information for the egress zone, so if the next route selected is on another gateway the CDR information retains the egress zone information from the previous attempt rather than being blank.

Steps to Replicate: 

1. With SBC GW-GW set up create two routes for a call such that:
      - route 1 is through local egress trunk group on the SBC1.
      - route 2 is through egress trunk group on the SBC2.
   - Set crankback profile to enable attemptRecordGeneration and add reason 41 (503 is mapped to 41 in default SIP to CPC mapping profile)
2. Send a valid INVITE to the SBC1 ingress that will first select route1 and route out through the egress trunk group on the SBC1. From the egress endpoint reject that call attempt with a 503 Service Unavailable.
3. The call will crank back and re-route using route2 through the SBC2 egress trunk group. From the egress endpoint reject that call attempt with a 503 Service Unavailable
4. Check that there are two Attempt CDR records for the call on the SBC1:
   1. ATTEMPT1 should have "Egress Zone Name" and "Egress Zone ID" fields populated correctly with zone information for zone of the SBC1 egress.
   2. ATTEMPT2 should have "Egress Zone Name" empty and "Egress Zone ID" as 0 because this is attempt for GW-GW route.

The code is modified to clear the internal CDR information for the egress zone when one route fails and the next route is selected.

Workaround: None.

Portfix SBX-104741: An error message is not displayed while creating an IP Peer, Ingress Ip Prefix, Radius Server.

Impact: The error message is not displaying while creating an IP Peer, Ingress Ip Prefix, Radius Server.

Root Cause: The error message was not displayed because we handled that error in the code and are unable to send it to the UI.

Steps to Replicate: Steps to reproduce the issue:

1. Login to the EMA as admin user.
2. Click on All Menu.
3. Expand Address Context > Zone > Ip Peer.
4. Create Ip Peer with IpAddress "::1" and ipPort "7080".
5. Click on Save.

The code is modified to throw an exception/Error to the UI.

Workaround: If we perform the same operation from CLI , we can get the same error message.

Duplicate audio entries in the route PSP.

Impact: If the number of codec entries in the packet service profile is more than 4, then the ERE was sending the codec entries 5-12 in the policy response twice. The duplicate entries could potentially cause problems when the SBC is performing codec intersection logic resulting in some codecs not being selected so the best audio match might not happen.

Root Cause: The ERE was incorrectly calling a function to populate entries 5-12 in the policy response twice.

Steps to Replicate: Populate the PSP with more than 4 codec entries in the ingress PSP and egress PSP and review the pes.logs to see the duplicate entries.

The code is modified to remove the functions that populated the codec entries 5-12 in the policy response so they only get return once.

Workaround: None.

Portfix SBX-102370: Subscription State is not updated as per expires parameter received in NOTIFY.

Impact: The subscription State is not updated based on the expired parameter received in the NOTIFY.

Root Cause: The SBC was not processing the expires in the Subscription-State header of the NOTIFY.

Steps to Replicate:

1. Run SUB-200 OK.
2. Send NOTIFY1 from UAS with expires parameter time t1.
3. Send NOTIFY2 from UAS with expires parameter time t2. (t2<t1)

The code is modified to consider the expires value in header, during the state "active" or "pending".

Workaround: None.

Portfix SBX-103669: Empty the "Egress Local Signaling IP Addr" field in the CDR.

Impact: When an incoming call is routed out of the SBC but is then rejected by the SBC with cause 132 Module failure before a backwards response message is received, the resulting CDR attempt has an empty "Egress Local Signaling IP Addr" field.

Root Cause: The SBC currently populates the "Egress Local Signaling IP Addr" CDR field when it processes a backwards response message for the call, and as a result, the field remains empty until a backwards response message is received and processed.

Steps to Replicate: 

1. Initiate a call by sending a valid INVITE to the SBC ingress TG to route towards the egress TG.
2. Once the INVITE is sent to egress by the SBC, put the signaling port to egress out of service.
3. The SBC will clear the call by sending a 503 message towards egress. The resulting attempt CDR should have valid address in "Egress Local Signaling IP Addr" field.

The code is modified to populate the "Egress Local Signaling IP Addr" CDR field when sending out the egress INVITE message.

Workaround: None.

Portfix SBX-102996: The wrong CSeq number was sent for BYE as part of a call termination after a SBC switchover.

Impact: The wrong CSeq number was sent for BYE as part of a call termination after an SBC Switchover.

Root Cause: When a In dialog Event INFO is received containing body/content type as 'media_control'. The SBC generates a new INFO request towards peer. Due to this the Cseq, in a dialog gets increased, this modified Cseq value was not mirrored to the SBY.

Steps to Replicate:

1. The call is established between PSTN User-B to Cisco User-A.
2. User-B initiates a Blind transfer call to PSTN User-C and answered on C.
3. Various INFOs are exchanged.
4. Perform SBC Failover after answering the call on User-C.
5. Send OOD Refer from Kandy Link having Refer-To header with method = BYE, valid Target-Dialog header and Request-URI points to SBC for Call Termination of PSTN User-C.
6. SBC sends BYE to both User C and User A.

The code is modified to the SBY during the INFO flow fixes the problem.

Workaround: Avoid the fail overs and use the stand alone set up.

Portfix SBX-73860: The SAM process coredumps after a restart during surrogate registration and deregistration.

Impact: The SAM process coredumps after a restart during surrogate registration and deregistration.

Root Cause: Accessing the pointer without a NULL check causes the core dump.

Steps to Replicate: 

1. Enable the flag sendSurrogateUnRegAfterReboot using set global signaling sipSigControls sendSurrogateUnRegAfterReboot enabled.
2. Enable surrogate registration the SBC starts sending REGISTER to registrar. Set the addressContext default zone zonein ipPeer PEER1 surrogateRegistration state enabled.
3. Registrar responds with 401 Authentication header response.
4. After receiving the challenge REGISTER that contains a valid Authorization header, the registrar responds with 200 OK.
5. Switchover from active to standby using request system admin HA-5100 switchover.
6. Switchover again using request system admin HA-5100 switchover.
7. Restart both the HA-PAIR.
8. Registrar responds with 200 OK for all the unREGISTERs sent by the active SBC when the services come up.
9. After successful deregistration, the SBC starts sending REGISTER to registrar.
10. Registrar responds with 401 Authentication header response.
11. After receiving the challenge REGISTER that contains a valid Authorization header, the registrar responds with 200 OK.

The code is modified to avoid a segmentation fault.

Workaround: None.

Saving changes to a SMM profile after doing an edit/update operation is taking long time[8 to 16 minutes].

Impact: A SIP Adaptor Profile with 250+ rules takes lot of time to create, including taking a lot of time to update.

Root Cause: When the SIP Adapter Profile with more than one rule is created, each rule is committed individually. However, after each commit EMA internally retrieves all the SIP Adapter Profiles along with their rules, though this is not required it is causing slowness of both create and update operation.

Steps to Replicate: 

1. Login to the EMA.
2. Navigate to the SIP Adapter Profile Screen.
3. Create a profile with 200+ Rules.
4. Click on Save button.

The code is modified to prevent the retrieval of SIP adapter profiles after each commit.

Workaround: The only workaround is to create the profile from the CLI.

Portfix SBX-100702: Observing the "ERROR : Channel 146 already in de-activated state" in the dsp log.

Impact: The dsp.log is overwhelmed with the "ERROR: Channel ## already in de-activated state" prints.

Root Cause: In the GPU design, as part of the command optimization, if there are more than one command for a channel in a particular time slot, the most recent command for the channel is sent to GPU for processing.

The call flow exercised, activates and de-activates the channel immediately, which could be potentially result in sending the activate command followed by de-activate command for the channel in the same time slot. Due to the command optimization mentioned above, the de-activate command is sent for the channel. Since the channel is already in de-activated state and an attempt is made to de-activate it, a message is printed.

Steps to Replicate: Activate and de-activate a channel immediately within 20ms.

The code is modified to disable the prints to not overwhelm the log and impact performance.

Workaround: None.

Portfix SBX-102745: The C_LIST macro for consistent memory was freeing.

Impact: When making changes to the local auth user configuration on the SBC, it was leaking small amounts of memory.

Root Cause: While processing the list of configured users, the application code was reading the CDB information into local buffers and then not freeing it, leading to a small memory.

Steps to Replicate: This issue is highlighted when make local user configuration changes in the engineering lab with ASAN images.

The code is modified with standard macros to delete the list structures in a generic manner to avoid this problem in the future.

Workaround: None.

The SBC is down and the standby registration with active failed, error 160004.

Impact: Standby is not allowed to join cluster and fails to start

Root Cause: The safplus checkpoint file is corrupt and the section needing to be overwritten is not found.

Steps to Replicate: The root cause of the checkpoint corruption is unknown/checkpoint corruption cannot be forced and therefore directly testing this fix is not possible.

The code is modified to re-add the missing section if it is not found.

Workaround: A complete outage is required in order to restart the active server.

The password padding with random characters by the SBC causes the RADIUS server to reject the password.

Impact: The radius password sent to the server has no zero characters at the end, following the password and a NULL.

Root Cause: The radius passwords are padded to 16 characters. The existing implementation did not set those padded characters to 0.

Steps to Replicate:

1. Configure a radius authentication server.
2. Use a password that is less than 15 characters long for the external radius user.
3. Set externalAuthententication to true.
4. Run a tshark session.
5. Login to the CLI.
6. Stop the tshark.
7. View the radius password element in wireshark after configuring the shared secret in wireshark under protocol preferences.

The code is modified so the padded characters are now set to 0.

Workaround: Use 15 or 16 character passwords.

The LI mediationServer signaling channel was online even if the interface was down.

Impact: Without this fix, the LI mediationServer signaling channel stays online, even after ipInterfacegroup attached to call data channel (CDC) is disabled post a switch-over.

Root Cause: The root cause came from code handling the LI mediation server signaling socket functionality did not register for ipInterfaceGroup operational status in standby mode.

As a result, post switch-over it does not get any notification when ipInterfaceGroup attached to CDC toggles. The signaling connection towards mediation server is not closed.

Steps to Replicate: 

1. Create the CDC and configure it for IMSLI and validate that the LI signaling channel is inService.
2. Run a switch-over.
3. Disable the ipInterfaceGroup attached to CDC.

The code is modified to register the ipInterfaceGroup that is attached in the CDC in standby mode as well so that if and once it becomes active due to switch-over, it is put into operational status of the ipInterfaceGroup attached to the CDC.

Workaround: None.

Incorrect indication for whether the SMM was applied or not required.

Impact: For machine parsing of trace logs, it is necessary to have a consistent field in the trace which indicates the message sent on the wire, regardless of whether the SMM is applied or not.

Root Cause: There was a design issue.

Steps to Replicate: 

1. Make a SIP-SIP call with L4 call trace enabled.
2. Messages sent on the wire have trace "SMM: OUTBOUND PRE-SMM"
3. Make a SIP-SIP call with L4 call trace enabled and an outbound SMM active.
4. Messages sent on the wire have trace "SMM: OUTBOUND POST-SMM"

When the SMM is applied to a message, the behaviour is unchanged.

When an SMM is not applied to a message, the L1-3 trace has "EXTRA_INFO: After SIPMM profile" and the L4 trace has "SMM: OUTBOUND POST-SMM".

When looking for messages as sent on the wire, the post SMM trace can always be used.

Workaround: None.

Portfix SBX-102501: The SBC fails to relay an embedded header in Contact of 3xx with statusCode3xx relay enabled.

Impact: The SBC fails to relay an embedded header in Contact of 3xx with statusCode3xx relay enabled.

Root Cause: The code required to send the embedded part in contact of 3xx was not present.

Steps to Replicate: 

1. Configure the SBC for A to B call.
2. Enable the flag statusCode3xx on egress TG IPSP.
3. Send an INVITE from UAC.
4. Send an 300 Multiple Choice from UAS with below Contact header that has embedded headers.

The code is modified to set only in the relay 3xx case to send the entire contact header transparently.

Workaround: None.

Portfix SBX-94798: MS Teams with FLRBT enabled hangs the call.

Impact: When Force Local Ringback Tone is applied on a call that been through SIP replace followed by SIP refer procedure, and the final trunk group has downstream forking enabled, the final 200 OK response may be queued indefinitely, meaning the call does not complete.

Root Cause: Interactions between Force Local Ringback Tone, multi-party and downstream forking.

Steps to Replicate: 

1. Make a SIP to SIP call.
2. A calls B.
3. B' replaces B.
4. B' refers to C.
5. Trunk for A has Force Local Ringback Tone applied.
6. Trunk for C has downstream forking enabled.
7. C responds with 180 with no SDP, 183 with SDP, then 200 OK.

The code is modified to prevent the queuing.

Workaround: None.

Portfix SBX-104826: The SBC fails to relay to 3xx and picks the next route when EnhancedLocalRedirection and StatusCode3xx flags are enabled.

Impact: The SBC fails to relay the received 3xx to the ingress side and picks the next route when EnhancedLocalRedirection and StatusCode3xx flags are enabled with 2 routes.

Root Cause: When both the StatusCode3xx and EnhancedLocalRedirection are enabled for the 2 routes scenario, performCrankback is set to true that leads to this issue.

Steps to Replicate: 

1. Configure the SBC for A to B call over ERE.
2. Configure ERE to return two routes for Initial call R1, R2.
3. Enable the flag statusCode3xx, EnhancedLocalRedirection on TG IPSP.
4. Send to INVITE from UAC.
5. Send to 300 Multiple Choice from UAS with Contact header and embedded headers.

The code is modified to ensure that performCrankback is not set to true when both the EnhancedLocalRedirection and StatusCode3xx are enabled.

Workaround: None.

A SIPREC Codec negotiation issue occured.

Impact: Without this fix, the wrong codecs are negotiated with the SRS when communication sessions is renegotiated.

Root Cause: Because of the wrong conditions in code, the codec information that is to be sent towards SRS is fetched from peer packet service profile rather than active service profile from the communication call leg.

Steps to Replicate: 

1. The SBC receives INVITE with G711A, G711U
2. Egress leg responds with A law and RTP stream is A law.
3. The SBC sends an INVITE to SIPRec server with G711A law and SIPRec servers responds with A law.
4. Egress leg sends re-INVITE and changes the order of codecs, G711U and G711A respectively.
5. The SBC relays the re-INVITE to ingress peer with G711U, G711A respectively and ingress peer responds with G711A.
6. The SBC was sending a re-INVITE with G711U codec to SIPRec Server.

With this fix, the SBC now sends a re-INVITE with G711A codec to SIPRec server.

The code is modified to fetch correct codec information from the communication session and the same is sent in recording session.

Workaround: None.

Portfix SBX-100864: The ASAN detected heap-use-after-free in DnsClientTcpSocketRecvMsg.

Impact: When a connection to a DNS server running on transport protocol TCP closes, memory is accessed after it is freed, potentially leading to unexpected behavior.

Root Cause: A connection to a DNS server running on transport protocol TCP closes.

Steps to Replicate: Address Sanitizer (ASAN) build is used to find the issue. Configure a DNS server with transportProtocol TCP. Run a call which requires DNS lookup.

The code is modified to correct management of DNS connections running on transport protocol TCP.

Workaround: None.

Portfix SBX-103814: The RECORDING CDR does not have correct value for SRTP field during a switchover.

Impact: The SIPREC related "RECORDING" CDR did not had correct values for SRTP related statistics fields during a switchover.

Root Cause: It was identified that there was a missing code that actually copies the SRTP related statistics fields data onto to the standby machine.

As a result, whenever the standby machine comes up as active the SRTP related data in RECORDING CDR was not populated correctly.

Steps to Replicate: 

1. Execute a SRTP SIPREC scenario.
2. Perform a switchover.
3. Verify the RECORDING CDR.

The code is modified to copy the SRTP related fields data onto standby machine.

Whenever standby machine comes up as active, the SRTP related data in RECORDING CDR is populated correctly.

Workaround: None.

The SCM process cores when targets set for OPTIONS/MESSAGE.

Impact: Without this fix, the SBC will core dump when the Out Of Dialogue messages like OPTIONS/MESSAGE are intercepted.

Root Cause: The double freeing of a data structure is causing the code dump.

Steps to Replicate: 

1. Provision packet cable 2.0 Targets corresponding to the out of dialogue OPTIONS that will be sent to the SBC.
2. Send OPTIONS to the SBC.

The code is modified to avoid double freeing of the corresponding data structure. Also addressed couple of memory leaks concerned with the same data structure in error cases such as ARS blacklisting.

Workaround: None.

Portfix SBX-76462: The incorrect MaxAverageBitRate value is populated in the "ingress codecParams1" after a switchover.

Impact: The incorrect MaxAverageBitRate value is populated in the "ingress codecParams1" after a switchover.

Root Cause: Some of the codecs attribute values were being set to 0 after a switchover, and as a result those values were getting assigned to the default values.

Steps to Replicate: Run aSILK_NB to SILK_NB call.
After the switchover, the MaxAverageBitRate value is 0 instead of 20000 in the "ingress codecParams1 and egress codecParams1".

To fix the issue, those statements are removed and change the attribute fields as a result.

Workaround: Some of the codes use certain attribute fields as being defined. But after the switchover, these used attributes field values have been changed to 0. For those codecs, these statements have been eliminated.

Portfix SBX-95677: The SBC is not feeding delayed RBT on monitoring failure in a late media pass through scenario in CLOUD ISBC and SWE ISBC.

Impact: The SBC is not feeding delayed RBT on monitoring failure in a late media pass through call

Root Cause: The fix for SBC not feeding delayed RBT on monitoring failure in a late media pass through scenario was applicable only when bToneAsAnnc flag is enabled.

Steps to Replicate: 

1. An INVITE received without SDP and no "PEM: supported" from UAC.
2. UAS sends 183 with SDP and no PEM having PCMU,AMR.
3. UAC send PRACK with SDP PCMU,G729 and UAS sends 200 OK for PRACK.
4. UAS sends authorized RTP.
5. UAS sends 180 without SDP and no PEM.
6. UAC send PRACK and UAS sends 200 OK for PRACK.

After an RTP Monitoring failure, the SBC should play the tone.

The code is modified to address the issue.

Workaround: None.

Portfix SBX-97598: The LeakSanitizer detected memory leaks at SipSgGetHpcCallProfileFromDb.

Impact: There was a memory leak when configuring and updating the HPC Call Profile gets the Strings Feature Code, Access Number and Number Translation.

Root Cause: Configuring and Updating the CLI Hpc Call Profile gets the Strings Feature Code, Access Number and Number Translation.

Steps to Replicate: None.

The code is modified so while de-allocating the list's while retrieving the data in the Feature Code, Access Number and Number Translation.

Workaround: None.

Portfix SBX-102985: The CDR field 'Ticks from Set up Msg to Service Est.' of a START record is getting logged with the wrong value.

Impact: The CDR field 'Ticks from Set up Msg to Service Est.' of the START record is getting logged with the wrong value when the "End To End ACK" is enabled and "No CDR Change in End To End ACK" is disabled.

Root Cause: Since the E2E ACK is enabled and No CDR Change in E2E Ack is disabled, the call connect time was not getting updated during the 200 OK and the value is still '0'. Due to this issue, the time difference of a call attempt time and the connect time was coming as a huge value.

Steps to Replicate:

1. Enable IPSP flag "End to End ACK" on both TGs.
2. Disable the flag "No CDR Change in End to End Ack".
3. Make a call from A to B and let the call be connected with session refresh enabled.
4. Once the session refresh occurs, disconnect the call and check the CDR.

The code is modified to not trigger the CDR start record when NRM call stable notification is received. Instead, the modified code delays the CDR until the ACK is received from the ingress, so that, the call connect time gets recorded before generating the start record and addresses the issue.

Workaround: None.

Portfix SBX-103807: The SBC disables the SRTP for audio when the EP disables and re-enables the audio.

Impact: The SBC disables SRTP for audio when EP disables and re-enables the audio.

Root Cause: During the modify offer processing, the SRTP value is taken from previous Active security PSP without checking if SRTP is valid or not.

Steps to Replicate: Set up Audio and Video RTP to SRTP pass through call between UAC-UAS:

1. UAC sends invite with Audio and video.
2. UAS responds with Audio and Video cryptoline.
3. UAC sends re-invite to disable Audio stream port=0 and inactive and Video with valid media port and sendrecv.
4. UAS responds 200OK with port=0, a=inactive and no crypto line for audio and video with valid port and crypto line.
5. UAC sends re-invite to add Audio stream back with valid media port and sendrecv and video with valid media port and sendrecv.
6. UAS responds 200OK with valid Audio and Video crypto lines.

Copy the Active security PSP only if the SRTP admin state is enabled to address the issue.

Workaround: None.

Portfix SBX-101264: The ingressBearerType and egressBearerType is "multimedia", even though negotiation is only with a single audio codec.

Impact: The ingressBearerType and egressBearerType is "multimedia", even though negotiation is only a with single audio codec.

Root Cause: At present, the bearerType is set as audio only if Audio stream is present at the first index. As a result, audio only call with audio stream present at index>1 (say (video port=0) + audio (valid port)) gets marked as multi-media call.

Steps to Replicate: 

1. Offer and answer contains video m line with valid port and audio m line with port 0.
   BearerType: Multimedia
2. Offer and answer contains video m line with port 0 and audio m line with valid port.
   BearerType: Voice
3. Offer and answer contains first audio m line with valid port and video m line with port 0.
   BearerType: Voice
4. Offer and answer contains first audio m line with port 0 and video m line with valid port.
   BearerType: Mutlimedia
5. Offer and answer contains multiple audio m-line.
   BearerType: Multimedia
6. Initial Offer answer with multimedia and then send re-INVITE with only audio.
   BearerType: Multimedia
   BearerType after Re-INVITE: Voice
7. Initial Offer answer with audio and then send re-INVITE with multimedia.
   BearerType: voice
   BearerType after Re-INVITE: Multimedia
8. Initial Offer and answer contains video m line with valid port and audio m line with port 0 then send re-INVITE with audio m line with sendonly (This scenario is as per SBX-97358) BearerType: Multimedia. BearerType after Re-INVITE : Multimedia

The code is modified to set the bearerType as audio irrespective of the audio stream index if the only active stream present in SDP is audio.

Workaround: None.

Portfix SBX-100868: The OAM does not push the configuration after a switchover to the standby OAM.

Impact: The configuration changes on the standby OAM is not rejected. After a switchover, it would not take the same configuration changes as it is already done locally. It will not playback either the save and activate as there is no playback logs created. There is no way to push the changes to the managed nodes until another switchover.

Root Cause: Skipping the playback log creation on the standby OAM caused an inconsistent state between the local configuration store and the playback logs.

Steps to Replicate: Perform a switchover with the same configuration to reproduce the issue.

The code is modified so attempts to make configuration changes on the standby OAM node are now rejected.

Workaround: Switchover back to the original active and restart the standby node so that it resyncs with the active.

Portfix SBX-100483: Observed a coredump Comp_SamP after disabling the sipSigPort.

Impact: While disabling the SIP signaling port, the SBC dumped core due to system error.

Root Cause: In the SBC, the wrong context ID was used to fetch the metavar related information while configuring (enabling/disabling) the SIP signaling port. As a result, the SBC has generated a system error.

Steps to Replicate: Bring up a Public Cloud (AWS) 1:1 HA system. Restart the standby SBC node. Try to disable the SIP signaling port. The SBC generated the coredump due to system error.

The code is modified to pass the correct context ID information while fetching the metavar related information while configuring the SIP signaling port.

Workaround: Execute the disabling of the SIP signaling port when the standby is up and running.

Portfix SBX-102927: The problem with migration of table_version_info was causing the EVS codec attributes non-readable after upgrade from 7.2 to 8.2.

Impact: The upgrade fails to set the default value for maxChannel flag.

Root Cause: The table_version_info table was not getting populated.

Steps to Replicate: The table_version_info table should have data on fresh install.

The code is modified to address the issue.

Workaround: The value in db for attribute3 in code entry table should be modified to set the default value of maxChannels.

Portfix SBX-93308: No DLRBT Tone Play can be heard on User-A phone in a Blind Transfer scenario using 180 ringing with SDP.

Impact: There is no DLRBT tone play heard on the transferee while transferrer does a blind transfer.

Root Cause: When 180 with SDP received from transfer target, SBC is trying to activate the tone resources towards transferee side. But, before this tone activation occurs, the CUT_THRU is received from the transferred leg that is causing to abort the tone.

Steps to Replicate: 

1. Make a call from the customer to the Cisco phone User-A that is a passthrough call.
2. Initiate the transfer call from the customer to the User B.
3. After a REFER (Direct Media) on the User-A phone DLRBT is not played.

The code is modified so it ignores this CUT_THRU received from the transferred leg and as a result, the tone is not getting aborted. When a CONNECT or RTP packet is received, then the tone is aborted.

Workaround: N/A

The SBC fails to send ACK for a self generated re-Invite when the E2E ACK and E2E re-Invite flags are enabled.

Impact: The SBC fails to send ACK for a self generated re-Invite when the E2E ACK and E2E re-Invite flags are enabled.

Root Cause: When both the E2E re-Invite and E2E Ack are enabled, do not send the ACK re-enable, the SBC was not handling ACk properly for the self-generated re-Invite.

There are scenarios when E2e is received RCVD from the Network where the ACK does not need to be sent for a re-Invite from the SIPSG in race condition scenarios.

So there was no check to identify if the re-Invite is self generated or the RCVD from a network and wrong logic was executed for the self-generated Invite.

Steps to Replicate: Test Configuration:
=============

1. Enable E2E ACK, E2E Reinvite, E2E PRACK, E2E UPDATE flags in IPSP.
2. Enable flag "send sdp in 200 ok if 18x reliable" in IPSP.

Test Procedure:
===========

1. UAC sends Invite with SDP.
2. UAS sends 183 with SDP reliable.
3. PRACK/200 OK is done.
4. UAS sends UPDATE with SDP.
5. UAC sends 200 OK with SDP.
6. UAS sends 200 OK without SDP.
7. UAC sends ACK.

The code is modified so if the re-Invite is self generated then send ACK.

Workaround: None.

Portfix SBX-99409: The EMA is not displaying all recordings for the SIPREC status command.

Impact: The proper SIPREC status was not displayed when more than one recording going on for a specific call on EMA or when checking the status with all the set of keys on the CLI. The SipRecStatus command was updated with new keys (GCID, legId, Recorder IP:port) but the application had a issue that it looked up the data only based on the GCID.

Root Cause: When the status command is issued through the EMA, the EMA queries with all the keys and as a result, the application would end up fetching the first recording for each of the queries. Same thing happens when querying on CLI with all keys for specific entry we end up getting first entry for that GCID.

As an example, when a quad recording is in progress on GCID1, when the SipRecStatus is queried using EMA, the EMA would query based on keys for each of the recording, say
Recording1: {GCID#1, Ingress-leg, IP1}
Recording2: {GCID#1, Ingress-leg, IP2}
Recording3: {GCID#1, Egress-leg, IP3}
Recording4: {GCID#1, Egress-leg, IP4}.
For all of the above keys, the application would display the data corresponding to the first recording.

Steps to Replicate: 

1. Set up the SBC with SIPREC Quad Recording for a single main call.
2. Check the status of the recordings on EMA.

The code is modified to receive all the keys of the SIPREC status and display the correct information.

Workaround: Check the status on the CLI without specifying all set of keys for the SIPREC status,

Portfix SBX-104705: There was a memory leak on the glusterfsd.

Impact: During a soak test on OAM, it was observed that the third party software glusterfsd process was leaking memory. If this occurred over a long duration, the OAM could run out of memory and a core would occur to that process.

Root Cause: The memory leak is observed for the glusterfsd process, which is the brick process. The use of the "gluster volume heal info" function causes the process memory usage to increase and not go down afterward.

Steps to Replicate: 

1. Ensure both Active and Standby OAM are running.
2. Monitor glusterfsd memory usage on both nodes.

Use the "gluster volume heal statistics heal-count" command to determine the gluster bricks heal state.

Workaround: Reboot OAM node OR Execute: gluster volume stop gvol1; gluster volume start gvol1.

Portfix SBX-737: The support for the 3072 bit long RSA keys is missing.

Impact: The SBC needs to support the 3072-bit RSA keys in certificates.

Root Cause: A feature change was required to support the 3072-bit RSA keys in certificates in addition to the 2048-bit and 4096-bit RSA keys.

Steps to Replicate: 

1. Test 3072-bit RSA key certificate on client. Make SIP-TLS call from the client to the SBC acting as TLS server.
2. Configure remote certificates with a 3072-bit RSA key.
3. Configure local certificates with a 3072-bit RSA key.
4. Test a 3072-bit RSA key certificate on the SBC acting as SIP-TLS server.

The code is modified for the 3072-bit RSA keys in certificates.

Workaround: None. 

Unable to access any log with the Platform Manager and EMA.

Impact: Unable to access log with the EMA and PM.

Root Cause: The jquery.ui.datepicker was not removed as this is updated in the jquery-ui.min file.

Steps to Replicate: 

1. Logged into the PM.
2. Go to Troubleshooting => Call trace/Logs/Monitors => Log Management
3. Select any categories to see logs.
4. Should be able to access logs.

Remove the jquery.ui.datepicker from the html file to address the issue.

Workaround: None.

Portfix SBX-102364: The SBC behavior is inconsistent with the isfocus parameter handling.

Impact: Issue 1: The SBC is not transparently passing the complete Contact header in 200 OK of SUBSCRIBE when there is a isfocus parameter.

Issue 2: The SBC is not adding the Record-Route in any message when doing a full Contact header transparency

Root Cause: Issue 1: The SBC passes contact header transparently only when the isfocus is present in request as well as response and would not send the contact header transparently only when the contact header in response has isfocus parameter.

Issue 2: The code to add a record route for notify and its response is not present.

Steps to Replicate: 

1. Configure the SBC for A to B call.
2. Enable the flag contactTransparencyForIsFocusMediaTag on both TGs.
3. Make a SUBSCRIBE-NOTIFY call flow.

Issue 1: The code is modified to check if the service bit is not set (for the response) and check if the contact header has the isfocus parameter, if the parameter is present, set SIP_SERVICE_TYPE_CONTACT_TRANSPARENCY_FOR_ISFOCUS_PARAM for a single contact scenario.

Issue 2: The code is modified to add the record route for notify and its response if isfocus parameter is present.

Workaround: N/A

Portfix PSX-35183: Setting the PSP with transcode conditional and then change it back to the TFT, all the settings from ‘conditionsInAdditionToNoCommonCodec’ are not reset back to the default values and in some cases affect call processing.

Impact: When the transcoder free transparency (TFT) is enabled and some of the ‘conditionsInAdditionToNoCommonCodec" flags are enabled too, the call processing may be different than when only TFT is enabled.

Root Cause: In the original design, flags of "Conditions In Addition To No Common" are
only displayed and configurable when PacketToPacketControl's transcode is Conditional. When a PSP's transcode is changed to other values from Conditional, all flags values would be preserved.

Steps to Replicate: 

1. Create a PSP, set transcode to Conditional.
2. Configure flags of "Conditions In Addition To No Common", making some enable and some disable.
3. Switch the transcode to "Transcoder Free Transparency".
4. Switch the transcode back to Conditional.
5. Check flags of "Conditions In Addition To No Common". They should be all disable.

The code is modified to set all flags of "Conditions In Addition To No Common" to disable whenever the PacketToPacketControl's transcode is set to "Transcoder Free Transparency".

Workaround: Before changing the transcode to "Transcoder Free Transparency".

1. Set the transcode to Conditional.
2. Set all flags of "Conditions In Addition To No Common" to disable.
3. Set the transcode to "Transcoder Free Transparency".

Portfix SBX-104990: In a secure call, the SBC does not increment port number in R-URI after processing Refer.

Impact: In a secure call with TLS configured, if the call is REFERed with a REFER-TO header containing a FQDN and port number, the SBC sends out new invite to the specified FQDN and port number. If that INVITE fails and the SBC then sends a subsequent INVITE on the next route it does not correctly increment the RURI port number for TLS.

Root Cause: The SBC code does not take into account that a re-route after a REFER-TO with FQDN and port number target needs to increment the port number for TLS, if the target after the re-route is different to the original target specified in the REFER-TO.

Steps to Replicate: 

1. With the recommended SBC configuration for MS Teams with TLS enabled between the SBC and MS Teams. Establish a call from PSTN to MS Teams.
2. Send a REFER from MS Teams witch includes following header -
   REFER-TO: <sip:sip2.pstnhub.microsoft.com:5061;transport=tls>
3. Based on the REFER, the SBC should route the call and send INVITE to sip2.pstnhub.microsoft.com using port 5061.
4. Reject this INVITE from MS Teams with a 503.
5. The SBC should then send an INVITE out using the second route to sip3.pstnhub.microsoft.com using port 5061.
6. Complete the call signaling and verify referred call is established correctly.

The code is modified to increment the RURI port number for TLS if doing a re-route to a target FQDN and port number that is different to the original target specified in the REFER-TO.

Workaround: None.

The SBC is not able to parse MSRP media packet 200.

Impact: When the SBC received "200" as MSRP response, it did not forward the same to other end as it considers 200 as a invalid response in absence of "OK" string in 200.

Root Cause: The SBC was strictly parsing "200 OK" for MSRP success response due to that if it is received "200" as response, it did not forward the same to other end.

Steps to Replicate: 

Test 1:

1. Configure the SBC for MSRP-B2BUA support.
2. Complete the INVITE signaling message for MSRP
3. Send a MSRP request message from UAC to the SBC, the SBC should forward this to UAS.
4. From UAS send "MSRP <transaction-id> 200" response message to the SBC.
5. The SBC should successfully send it to UAC.

Test 2:

1. Configure the SBC for MSRP-B2BUA support.
2. Complete the INVITE signaling message for MSRP
3. Send a MSRP request message from UAC to the SBC, the SBC should forward this to UAS.
4. From UAS send "MSRP <transaction-id> 200 OK" response message to the SBC.
5. The SBC should successfully send it to UAC.

The code is modified to consider 200 as valid MSRP response.

Workaround: There is no workaround at the SBC for this issue, UAC/UAS need to send MSRP response as "MSRP <transaction-id> 200 OK" to avoid this issue.

Portfix SBX-100122: There were various display issues related to media port range modifications.

Impact: Issue 1,2,3: The CLI display of mediaPortRangeUnassigned to TGs was not regulated by the system media port range setting, causing the mediaPortRangeUnassigned CLI command displaying the wrong ports.

Issue 4: If the basePorts for different TGs are the same in a same zone, their maxPorts will be added up in CLI display of mediaPortRangeAssigned. tcpPortRangeAssigned has the same problem.

Root Cause: Issue 1,2,3: The root cause was that the SBC had not considered the system media port range setting as restriction when calculating unassigned ports.

Issue 4: The root cause was that assigned media port range map had not been indexed by TG, but by basePort had been a mistake.

Steps to Replicate: While working on the SBC CLI

1. Show configuration details system media mediaPortRange
2. Show table addressContext addressContextName ipInterfaceGroup IG_name mediaPortRangeUnassigned
3. Modify configuration details system media mediaPortRange to much smaller range.
4. Repeat step 2, to make sure that mediaPortRangeUnassigned gives ports always within the range of "show configuration details system media mediaPortRange"
5. Create multiple TGs in each zone. Assign mediaPortRanges to these TGs. Some range with the same basePort in the same zone. Some with different basePorts in the same zone.
6. Repeat step 2 to make sure that mediaPortRangeUnassigned gives ports always within the range of "show configuration details system media mediaPortRange"
7. Show table addressContext addressContextName ipInterfaceGroup IG_name mediaPortRangeByTGNameAssigned / mediaPortRangeUnassigned. Ensure that the outputs are correct.
8. Repeat steps 2-7 with tcpPortRangeByTGNameAssigned / tcpPortRangeUnassigned
9. Repeat the display steps above in EMA.

The code is modified for the following issues:

1. Added system wide media port range into SBC SIPSG cache, so that the restriction will be checked before display of mediaPortRangeUnassigned.
2. Changed the map of the assigned mediaPortRange to indexed by TG name, not basePort any more.
   Therefore, the CLI/EMA syntax becomes mediaPortRangeByTGNameAssigned/ tcpPortRangeByTGNameAssigned, instead of mediaPortRangeAssigned / tcpPortRangeAssigned.

Workaround: It is a display issue, does not impact call.

There was a No Way Video after A/V calls is resumed after hold.

Impact: There was a No way video after a audio video call is resumed using a late media re-INVITE.

Root Cause: The SBC does not send "sendrecv" in 200 OK for late media re-INVITE, and instead sends the same datapath mode that was negotiated last, which could be inactive.

Steps to Replicate: Set up a audio and video call. Put the call on hold by sending a=inactive in audio and video SDP. Then send a late media re-INVITE to take the audio off-hold. The SBC will send SendRecv for audio and video in 200 OK and let the remote end choose if it wants to remain on hold or come out.

The code is modified to send a sendrecv for other streams when the audio in call is being resumed aftera  hold.

Workaround: Do not use late media re-INVITE to come out of hold.

Portfix SBX-105262: The SBC is sending unexpected Re-INVITE to egress side in SRTP early media scenario.

Impact: The SBC is sending unexpected Re-INVITE to egress side in SRTP early media scenario.

Root Cause: This issue is caused because of a side-effect of SBX-103807.

Steps to Replicate: Call flow:

1. UAC sends Invite with 100 rel required.
2. UAS sends 18x with SDP+SRTP (SHA-1-32).
3. Ingress sends PRACK with SDP+SRTP (SHA-1-32).

Copy the active security PSP only if audio stream is present to address the issue.

Workaround: None.

Portfix SBX-105114: Usage of the kill command output in active and standby CE_node logs.

Impact: The kill command usage gets printed in the CE_Node log file of the managed nodes.

Root Cause: No glusterfs process is present when the kill command is executed by the glusterSetup.sh script called by sbxConfigUpdater.sh.

Steps to Replicate: 

1. Reboot the Standby OAM.
2. Ensure that kill usage does not appear in the CE_Node log file of the managed nodes.

Redirect kill command error output to /dev/null to address the issue.

Workaround: None.

Portfix SBX-104360: The SWITCHOVER ACT Records are not generated in the SBC with the HA mode set as Nto1.

Impact: On an N to 1 system, the SWITCHOVER record is not written to the accounting file on the new active node after a switchover.

Root Cause: The SWITCHOVER record is sent to the ENM process before it has opened the accounting file, so the record is not written.

Steps to Replicate: 

1. Perform a switchover a system.
2. Check that the switchover record is written to the accounting file.

The SWITCHOVER record is stored and then written out when the accounting file is opened to address the issue.

Workaround: None.

Portfix SBX-104236: Our SIPREC implementation is prone to failure to record a stream and to tear down the SRS call leg when SRS (SIP Recording Server) sends a re-INVITE to SBC.

Impact: SIPREC handling had race conditions issues in the scenario where a RE-INVITE was received on the main call and also RE-INVITE was received from SRS server at the same time.

This resulted in recording failure and the SBC sent out BYE towards SRS.

Root Cause: This race condition of RE-INIVITE from the main call end point and SRS server simultaneously was not handled on the SBC.

Steps to Replicate: 

1. Make a SIPREC call.
2. Send RE-INVITE with codec change/hold on main call (either from UAC or UAS) this triggers a RE-INVITE towards SRS.
3. SRS also sends RE-INVITE at the same time when the SBC receives 200 OK for main call RE-INVITE.

The RE-INVITE triggered/sent towards SRS due to the main call RE-INVITE is queued in the situation where the SIPREC leg is in middle of processing the RE-INVITE that was received from the SRS. The queued RE-INVITE is sent once the RE-INVITE transaction received from the SRS was completed.

Workaround: The SRS can avoid sending an immediate hold RE-INVITE if no media is observed on SIPRec leg.

There was a SCM coredump observed when multiple gateway TGs are created in a GW-GW call on a HA set up.

Impact: On a HA set up, the Standby box SCM Process dumps core when the Standby starts to sync from Active, post a switch over, and the switch over occurs after a scenario where the Gateway TG's are created and then a GW TG with a lower index (Created earlier) is deleted.

Example:

1. Create GWTG1, GWTG2 and GWTG3.
2. Delete the GWTG2
3. Run a switch over.

Root Cause: The coredump is caused due to difference in indexing of gateway TG's in active and standby boxes.

The GW TG indexes were out of sync between the Active and Standby.

The Active SBC had holes in the indices of the GW TGs after deletion.

The standby SBC does not have holes for the GW TGs that are present post deletion and occupy different index when compared to active.

Steps to Replicate: 

1. HA set up with GW-GW configurations.
2. Create 3 Gateway Trunkgroups: GWTG1, GWTG2 and GWTG3.
3. Delete the GWTG2.
4. Perform a switchover.

The code is modified to ensure that the GWTG indexes on the Active and Standby are in sync.

Workaround: None.

The MCF sent back to ingress as the MPS.

Impact: Some fax T.38 endpoints send entire DCS V.21 signal in one packet as opposed to 1 octet per packet. This can causes fax failures.

Root Cause: A burst of octets in a single packet is essentially a burst, and causes potential problems as these packets get queued for modulation and cause delay and later TCF (high speed) signal some packets get dropped.

Steps to Replicate: This issue was reproduced with a T.38 capture from field that contains a DCS followed by a TCF and a page.

Without a fix, the high speed modulated page signals starts and then, there is a gap and it restarts. (Picture is loaded in broken_page.png).

After the fix, the high speed modulated page signal is correct, it does not show het gap and restart behavior for 7.2.

The code is modified to accommodate larger bursts.

Workaround: This bug is a specific interoperability problem with endpoints that exhibit burst single packet DCS signals. For such endpoints, we do not have a workaround, unless there is some configurations are available on those devices to modify this behavior.

Sending m=application 0 UDP/BFCP (null) in case of UPDATE.

Impact: The SBC sends m=application line in incorrect format in SIP UPDATE message.

Root Cause: The SBC does not format the m=application line for UDP/BFCP when sending UPDATE message.

Steps to Replicate: Test case 1: Reproduce the issue:

The SBC sends an INVITE with SDP(with audio and video lines and m=application) and receives rel 18x with SDP (with audio and video lines only) followed by the final 200OK with SDP (with audio and video lines and m=application) if flag is enabled then SDP of 200OK should be ignored by the SBC.

UPDATE message content:

m=application 0 UDP/BFCP (null)

Test case 2: Verify the issue the same scenario as test case 1.

UPDATE message content:

m=application 0 UDP/BFCP *

The code is modified to ensure the SBC sends the m=application line in correct format.

Workaround: None.

There were coverity issues in OAMNODE.

Impact: When processing a show list command under the node branch from the OAM node, if the target node fails to read the command path in the request, the code will access memory immediately after freeing it. While in most cases this should not cause issues, accessing memory after it is freed is not good behavior and could result in unexpected behavior and potentially in the worst case coredumps.

Root Cause: Error handling flow in the code is incorrect. It hits some code for the success flow.

Steps to Replicate: Run coverity tests to reproduce the issue.

The code is modified to address the issue.

Workaround: None.

Portfix SBX-105361: The SSH access was not re-enabled on SBC 5400 for the linuxadmin after an upgrade.

Impact: The SSH is enabled for root user in SBC5400 after LSWU or PM upgrade.

Root Cause: The getVirtualType function in sonusUtils.sh incorrectly set the virtual type on the SBC5400.

Steps to Replicate: Enable the SSH and run the LSWU, the SSH should not enable for root user.

Validate the previous virtual type in getVirtualType function to address the issue.

Workaround: None.

Portfix SBX-105270: There was a cpxAppProc leak for a MRFP call.

Impact: There was a small memory leak occurs on SBC/MRFP nodes when action/status/stats under the node branch is accessed through the OAM node CLI or EMS.

Root Cause: Resource references are cleared mistakenly after serving the request from the OAM node.

Steps to Replicate: Execute a node branch command repeatedly and monitor the CPX process size on the target node.

Resource references are cleared only when the system is shutting down to address the issue..

Workaround: Avoid using node branch commands in automated/periodic operations. Manual use should be ok as the leak is small.

Portfix SBX-102700: Number mapping (translated, RDI, To hdr, R-URI) is odd.

Impact: The SBC is not adding translated number received from PSX to RequestURI when Diversion header is present in the ingress Invite.

Root Cause: If ingress Invite does not contain Diversion header, then SBC is adding translated number to RURI. So same result is expected when diversion header is present.

Steps to Replicate: 

1. Make an SIP call by sending an Invite with Diversion header from UAC.
2. Configure the PSX to return "translated number" for the called party number in policy response. The SBC includes the translated number in Request URI and routes the call.

The code is modified to populate the RequestURI with the translated number even when the Diversion header is present in the initial Invite.

Workaround: None.

SBX-103222

2

The SBC is not sending an INVITE to the MRF for a modified transcode call.

Impact: Run a MRF call flow by configuring the MRF profile.

Root Cause: The MRF Profile values are not correctly read by the application.

Steps to Replicate: 1. Configure the System for doing the MRF call flows.
2. Try the MRF call flow, the MRF must be invoked.

The code is modified to read the values from a proper MRF path.

Workaround: The sbxrestart should solve the issue.

Portfix SBX-106004: The EMA display error when SMM deleted through the CLI.

Impact: The EMA display error when SMM deleted through the CLI.

Root Cause: In the EMA, we are checking the ordering of the rules. If the Order is not continous then we are throwing the error.

Steps to Replicate: 

1. Log in to the EMA
2. Navigate to Profiles > Signaling > SIP Adaptor Profile.
3. Select one SMM rule and try to edit it.
4. There we do not found the error if the rules are not continuous.

The code is modified to address the issue.

Workaround: None.

The IPv4 Path MTU Discovery (RFC1191) not working for the UDP.

Impact: The SBC uses the MTU of the outgoing network interface as the Path MTU when sending out SIP-UDP packets.

Without the fix, the SBC does not update Path MTU properly even if a router in the path sends "ICMP Destination Unreachable Fragmentation Needed" message with smaller Path MTU value. The large packets requiring fragmentation to fit the Path MTU from the SBC will not reach the SIP peer.

Root Cause: While handling "ICMP Destination Unreachable Fragmentation Needed" error message for UDP packet, route/fib lookup did not consider ipInterfaceGroup and addressContext. This resulted in not updating Path MTU of the proper route entry to the peer for sending SIP-UDP packets.

Steps to Replicate: The SBC sends SIP-UDP packets of length 1500 bytes toward the SIP peer when a path to the peer has a smaller MTU. A router with the smaller MTU sends "ICMP Destination Unreachable Fragmentation Needed" to the SBC.

Without the fix, the SBC keeps on sending SIP-UDP packets, and the router keeps on sending "ICMP Destination Unreachable Fragmentation Needed" to the SBC.

The SBC, with the fix, adjusts the Path MTU, performs the fragmentation on SIP-UDP packet and sends fragments to the SIP peer.

The code is modified to add ipInterfaceGroup and addressContext information in handling ICMP Destination Unreachable Fragmentation Needed message in the Linux kernel.

Workaround: Disable Path MTU Discovery by setting ipNoPmutDisc value to.

admin@SBCSWE03% set system admin sbcswe03 kernelParams ipNoPmtuDisc 2
admin@SBCSWE03% commit

Note: The support for ipNoPmutDisc was added to 7.2.5R000, 8.2.4R000, and 9.2.0R001.

Portfix SBX-103986: There was an incorrect RTP time stamp in the SBC packet capture.

Impact: After an SBC upgrade, RTP/RTCP time stamp in the SBC packet capture shows the year 2036.

Root Cause: Present logic was not considering the edge cases while checking the last modification time of the NTP log.

Steps to Replicate: 

1. Configure a remote syslog server:
   set oam eventLog platformRsyslog linuxLogs ntpLog enabled
   set oam eventLog platformRsyslog servers server1 port 10514 protocolType udp remoteHost <remote host ip>
   com
   set oam eventLog platformRsyslog syslogState enabled
   com
2. Create a second dummy NTP server (just to trigger some logs to be written to ntp.log):
   set system ntp serverAdmin 1.2.3.4
   com
3. Delete the dummy NTP server:
   del system ntp serverAdmin 1.2.3.4
   com
4. Wait at least several seconds.
5. Repeat steps 2 and 3.

The code is modified for the edge case of NTP log last modification time.

Workaround: None.

AddressSanitizer error: heap-use-after-free on the address 0x6190000ba218 at pc 0x56473eecd69f bp 0x7f27682167e0 sp 0x7f27682167d8

Impact: The heap use after free error while running the PC2 ingress leg interception on OPTIONS/200OK call SBC_8.2x ASAN Build.

Root Cause: For the 4C, is using memory that is free by previous code. For the 6, the leak due to a missing null check.

Steps to Replicate: 

1. Create a ASAN Build.
2. Set up a PC2 and run a call.
   

The code is modified in the following manner:

• Added a calling function before freeing the memory.
• Added a NULL check.

Workaround: None.

The PSP qos value msrpDscp non-zero causes a PES process crash.

Impact: The PES process crashes when the apps are starting up and if msrpDscp in the PSP QoS values are configured with non-zero.

Root Cause: A debug statement tried to print a integer as a string, causing memory problem.

Steps to Replicate: 

1. Configure a non-zero msrpDscp value.
2. Restart the SBC.

Before the code change, the PES will crash.

After the code change, the PES will not crash.

The code is modified to print integer as integer.

Workaround: None.

The debug command caused the SCM process to core dump.

Impact: We are hitting a Healthcheck timeout when displaying the output of the following debug command:
"request sbx sipsg debug command "svcgrp -ce 0 -scm <x> -1""

Root Cause: The Heathcheck timeout error occurs due to taking too long to display the data for all of the service groups when there are very large number of trunk groups configured.

Steps to Replicate: 

1. Configure the 1000 TGs
2. Issue the following command:
   "request sbx sipsg debug command "svcgrp -ce 0 -scm <x> -1""

The code is modified to only display up to 50 service groups in order to prevent this Healthcheck timeout.

Workaround: To prevent this issue, avoid using the following command if there are large number of TGs configured:
"request sbx sipsg debug command "svcgrp -ce 0 -scm <x> -1""

Portfix SBX-102478: An Automated Upgrade failure for the SLB/SBC from 9.0 to 9.1.

Impact: The newly upgraded OAM VM did not properly boot. On the investigation, it was seen that the Cluster IP was not set correctly, even though an upgrade was received from VNFM, which contained the correct Cluster IP.

Root Cause: The updated userData.json received from VNFM was overwritten with the initial boot userData.json (that did not contain the Cluster IP), even though there is code in convertVnfm.py specifically to prevent that from happening. On investigation, it was seen that a bad propagation of fixes from other releases into main introduced an indentation problem. Indentation is significant in python, and the bad indentation made the code that checked for an updated userData.json fail to work properly.

Steps to Replicate: Use different types of auto-upgrades should be tested, because they seem to expose the problem with non-updated userData.json more easily than a normal orchestration.

The code is modified so the correct indentation is restored.

Workaround: If the orchestration is failing without this fix, a restart of the VM should be manually triggered to have it re-run the code in question. This needs to be done before VNFM fails the upgrade (1 hour).

There is packet verification improvements for T.38 stack.

Impact: In some of the crashes from the field (e.g. SBX-101155), we have observed a frame in jitter buffer that has unreasonably large fld_cnt as compared to the size of the packet itself. This is a malformed packet and such packets may lead to crashes.

Root Cause: These malformed packets are the root cause.

Steps to Replicate: TBD

The code is modified to look for UDPTL packet size and frame size inconsistencies and to allow packets with inordinate field count to be accepted by the stack. This logic is improved to reject such packets.

Workaround: None.

Special characters in the DN hinders the EMA display.

Impact: Special characters in DN are throwing error in the EMA.

Root Cause: Special characters were not supported for the DN through the EMA UI.

Steps to Replicate: 

1. Create Route with a special character in DN.
2. Save/Edit/Delete the Route.

The code is modified to address the issue.

Workaround: None.

Restoring the backup SMM configuration is failing.

Impact: Restoring the backup configuration was failing.

Root Cause: This issue is caused because the loadConfig was failing because of errors in HwModuleServer.cpp file.

Steps to Replicate: Run a load configuration from the CLI, and configure the load properly.

The code is modified to address the issue.

Workaround: None.

The reenableOsAccount silently sets the account expiration to 30 days.

Impact: Using the reeanbleOsAccount will add account aging to any OS account.

Root Cause: The logic does not take into account state of OS account aging or if OS account aging should be applied (e.g. root).

Steps to Replicate: 

With the OS account aging enabled:

1. Test the root:
i. Set the reeanbleOsAccount for user root.
ii. Check there is no aging is applied at OS level: chage -l root.

2. Test the Confd user:
i. Create user through CLI.
ii. Set the reeanbleOsAccount for this CLI user.
iii. Check there is no aging is applied at OS level: chage -l <CLIUser>.

3. Test the OS user with a password:
i. Create the OS user with a password.
ii. Disable and then Enable OS account aging state.
iii. Set the reeanbleOsAccount for this OS user.
iv. Check the correct aging is applied at OS level: chage -l <OSUser>.

The code is modified to check in the OS account aging is enabled and is applied.

Workaround: None.

The Control Display of Table Columns is not working for the DiamNode EMA page.

Impact: The Control Display of Table Columns is not working for the DiamNode EMA page.

The options selected by the user were not retained and maintained properly when the user navigates to some other page

Root Cause: Missing Null Checks on a userName parameter while updating the columns in DiamNode Page.

Steps to Replicate: 

1. Successfully verify the loading of the EMA Application.
2. Successfully verify the rendering of the DiamNode Page.
3. Successfully verify that the columns are displayed according to the show/hide logic set in the page.

The code is modified to ensure that exceptions are handled appropriately.

Workaround: None.

The SBC was sending an INVITE with incorrect b= line values.

Impact: The SBC sends the wrong RR and RS values in the outgoing INVITE for a transcoded call.

Root Cause: On the D-SBC, the NrmaComputeOfferPspForAnswerLeg() gets executed twice when intersecting. Once with ingress route PSP and once with egress Route PSP and even though we pass the right route PSP to the function, the rtpOptions passed to the function is wrong that results in the wrong computation of RR and RS values.

Steps to Replicate: The SBC is configured to make a SIP-SIP call procedure:

1. Configuration: Egress PSP: AMR (WB) Bandwidth efficient. Ingress PSP: AMR (WB) Bandwidth efficient.
2. RTCP is enabled only on egress PSP and RR/RS bandwidth is configured as 500.
3. Flag 'Send RR/RS in SDP' is enabled in IP signaling profile for Egress.
4. Initiate a SIP-SIP Audio call with the Audio codec as AMR WB without RTCP bandwidth related parameters (RR/RS not present in SDP).
5. Observe the SBC sending wrong RR and RS values in outgoing INVITE.

The code is modified to pass the correct rtpOptions for computing the RR and RS values.

Workaround: None.

The SBC was sending AMR-WB twice and the SBC was accepting codec in answer that is not present in offer

Impact: In a Late Media Call with HD Codecs configured in the Route PSP, the SBC was sending 2 HD codecs in its offer towards the Egress even when the "preventSameCodec" configuration flag is enabled in Trunk Group.

Root Cause: The SIPSG module was not passing the "preventSameCodecTransCode" flag to NRMA for Late Media calls.

Steps to Replicate: The steps cannot be replicated.

The code is modified to now pass the "preventSameCodecTransCode" flag for late media calls. For normal media calls, this is working in 8.2.3 releases

Workaround: None.

Portfix SBX-100320: There were SBC 9.1 coverity issues.

Impact: In several modules, coverity pointed illegal access of NULL pointer, that may cause a crash in code.

Root Cause: Illegal use of pointers, without checking whether they are NULL or not.

Steps to Replicate: After running the coverity, the illegal access of pointers were not encountered again.

The code is modified to correctly check the pointer, whether it is NULL or not. If it is NULL, then no further processing of that pointer is done

Workaround: None.

LSWU. Call/Registration Data syncInProgress.

Impact: Issue#1: Upgrade failures with EDNS Feature

The LSWU Upgrade from the SBC from pre 8.2 release to a 8.2 or higher release fails when EDNS servers are configured.

When the ednsServer statistics table has non-zero values for the field “ednsFailures”, the SBC synchronizes the stats structure. Since the corresponding structure does not have serialization support, the sync is aborted and the upgrade fails. The ednsFailures stats are updated when the SBC encounters RCODE 1, 2 and 4 errors for EDNS qeueries.

Example pre-8.2 configuration causing this issue:

admin@PLUM> show status addressContext default dnsGroup DnsGrp
dnsServerStatistics 2
{ ipAddress 10.xx.xx.xx; queries 5; timeouts 0; errors 2; referrals 0; totalTcpConnection 0; tcpConnectionFailed 0; tcpConnectionSuccess 0; tcpConnectiontorndown 0; tcpFallback 0; ednsStatus supported; ednsFailures 1; }
[ok][2020-11-06 04:07:27]

Issue#2
The EDNS Server stats is lost during an upgrade from 8.2/9.0/9.1 to 9.2 because of an incorrect serialization code.

Root Cause: Issue#1: Upgrade failures with the EDNS Feature

There are multiple issues in serialization support for DNSC_REDUND_MIRROR_EDNS_SERVER_STAT_INFO_CMD_MSG_STR

1. The DNSC was not registered for Serialization till 8.2. The DNSC was registered for serialization support in 8.2.0R0 with the feature SBX-75865.
2. The postUnpack methods for the structures had wrongly updated the length of the message. This was also fixed in 8.2.0R0 with the feature SBX-75865.

Issue#2: The EDNS Status was lost
The wrong datatype was used for serializing DNSC_REDUND_EDNS_SERVER_STAT_DATA, i.e. VariableSizeOfVariableDataArrayOperations instead of VariableSizeArrayOperations.

Steps to Replicate: Procedure:

1. Configure a SBC HA pair for basic call with 8.2 version.
2. Configure the DNS Server with eDNS Monitor enabled and a FQDN IP Peer.
   set addressContext default dnsGroup DnsGrp ednsSupport enabled
3. Make a call for same FQDN and send RCODE error(1,2 or 4) from DNS server.
4. Check the DNS statistics.
   show status addressContext default dnsGroup DnsGrp dnsServerStatistics
5. Upgrade the Standby box followed by active box with post 8.2 SBC version.
6. Upgrade should be successful and DNS statistics should be proper.

The code is modified to handle the LSWU Upgrade of SBC from pre 8.2 release to a 8.2 higher release.

Workaround: Issue#1: Upgrade failures with the EDNS Feature

Before upgrading a box that is older than 8.2R0, the following commands on all the DNS Groups needs to be issued before we upgrade.

Note: The ednsServer stats is lost as a consequence during the upgrade.

1. Issue a command to clear the EDNS statistics for all DNS Groups in the system.
   admin@PLUM> request addressContext default dnsGroup DnsGrp dnsServerReset
   reason DNS Server statistics are Reset
   [ok][2020-11-06 04:08:13]
   admin@PLUM> show status addressContext default dnsGroup DnsGrp
   dnsServerStatistics 2
   { ipAddress 10.xx.xx.xx; queries 0; timeouts 0; errors 0; referrals 0; totalTcpConnection 0; tcpConnectionFailed 0; tcpConnectionSuccess 0; tcpConnectiontorndown 0; tcpFallback 0; ednsStatus supported; ednsFailures 0; }
   [ok][2020-11-06 04:08:22]
   admin@PLUM>
2. Disable the ednsSupport to stop mirroring
   set addressContext default dnsGroup DnsGrp ednsSupport disabled

Issue#2: The EDNS Stats was lost.

No workaround. 

The high CPU Utilization was observed on GPU UXPADs with 1152 chans/process.

Impact: The transcoded calls using FMTD (Fax and modem tone detection) require most CPU cycles than expected. As a result, in a loaded system higher than expected CPU utilization is observed.

Root Cause: Root cause for this is that the FMTD module code was checked in with a debug compiler flag. The debug flag results in code is less optimal and consumes higher number of CPU cycles.

Steps to Replicate: This requires a use of high load to observe the problem. Typical with GPU use, the expected numbers are about 18432 G729A-G711U calls with a max UXPAD utilization to be 80-81%. Due to this bug, 90% CPU utilization is observed with about 13k calls. As a result, the test steps are:

1. Run a high load of transcoded calls with fix and observe CPU utilization.
2. Run a high load of transcoded calls without fix and observe CPU utilization.

With a fix, the utilization should be same as that for 8.1.0R6 as a comparison.

The code is modified to make a file for the FMTD module to remove the debug flag and then rebuild the optimized library.

Workaround: None.

The SBC was terminating the call when a 491 response is received for the self generated Re-INV.

Impact: The SBC is terminating the call when a 491 response is received from a Peer for the self generated Re-INV.

Root Cause: When a 491 response is received from the peer for the self generated Re-INV, the SBC is treating this as an error response and as a result, generates a internal clear event and the call is getting terminated.

Steps to Replicate: IPSP flag configuration:

DML flag disabled.
relayStatus4xx6xx flag enabled.
Create a transparency profile and attach it to the egress TG.

1. Make a call from user A to user B with offer containing multiple codec towards user B.
2. Let user B answer with multiple codecs, such that, the SBC triggers a media lock down Re-INV.
3. Let user B send 491 Request Pending.

The code is modified so that the call will not get terminated.

Workaround: None.

The search filter tab for Routing page in EMA is not working properly in 8.2.1R1.

Impact: When user try to perform search for # % & _ in filter tab of EMA route page, results were not fetched as expected.

Root Cause: The search values are not encoded before making a request to get results.

Steps to Replicate: 

1. Navigate to the Route screen in the EMA.
2. Entered values with special character to perform a search.
3. Received the result as expected.

Encoding the values before making the request to get exact results addresses the issue.

Workaround: None.

Portfix SBX-101057 (Originated in Release 7.2.5): The iceSupport and SMM iptgStore coredumped.

Impact: The iceSupport and SMM iptgStore configuration, and an invalid incoming call may caused the SBC to core.

Root Cause: After a response failure to the invalid call, the SBC still tries to initiate set up call. As a result, it accesses the invalid address.

Steps to Replicate: Configure the iceSupport and SMM storeIPTG, incoming call with invalid candidate (no UDP).

The code is modified to not try to initiate set up call if the call fails.

Workaround: Use the SMM to drop an incoming call.

There was a switchover after the TLS certificates were renewed.

Impact: When certificate is modified, an old certificate is freed and new certificate is updated. But after freeing the old certificate, the pointer is not set to NULL that can cause undefined behavior due to an invalid access.

Root Cause: The issue is when the customer modified the certificates through the CLI, it resulted in the SAM process to core. This is due to memory corruption.

Steps to Replicate: Changing the TLS certificates when the TLS load is running will result in memory corruption and the core can occur any time if there is active TLS load running and if the existing TLS calls try to access the certificates that were deleted already.

Set the pointer to NULL so that there is not invalid access to the pointer.

Workaround: N/A

The SBC SWe was not generating a switch over trap-sonusCpRedundGroupSwitchOverNotification.

Impact: A switchover notification trap may not be generated.

Root Cause: The event, used to generate the trap, is incorrectly dropped due to the logic that thinks the event is stale.

Steps to Replicate: Perform switchover and ensure the trap is properly generated.

The code is modified to improve the event handling so that the necessary events process and do not mistakenly drop.

Workaround: The problem only occurs during slow processing. Avoid anything that can slow processing down like info level logging, or any external influences on the disk.

There was an issue with the SBC 7000 TCP window size. 

Impact: Add the CLI support to set kernel parameter net.ipv4.tcp_window_scaling

Root Cause: In certain SBC deployment scenarios (a large number of devices running SIP-TLS), tcp_window_scaling caused small TCP window size.

In such cases, the customer had to use Linux command to disable tcp_window_scaling on each SBC node.

Steps to Replicate: 

(1). Show initial setting for tcp_window_scaling on both active (SBCSWE01a) and standby (SBCSWE01b)
[root@SBCSWE01a ~]# cat /proc/sys/net/ipv4/tcp_window_scaling
1
[root@SBCSWE01b ~]# cat /proc/sys/net/ipv4/tcp_window_scaling
1
[root@SBCSWE01b ~]#

(2). Use CLI to change the kernel param value and display the changed value on both active and standby.
admin@SBCSWE01a% set system admin SBCSWE01 kernelParams tcpWindowScaling disable
admin@SBCSWE01a% commit
Commit complete.

[root@SBCSWE01a ~]# cat /proc/sys/net/ipv4/tcp_window_scaling
0
[root@SBCSWE01a ~]#

[root@SBCSWE01b ~]# cat /proc/sys/net/ipv4/tcp_window_scaling
0
[root@SBCSWE01b ~]#

The code is modified to fix the issue and the Linux command is no longer needed with the modified code.

Workaround: Use the Linux command to set the kernel parameter on both active and standby.

[root@SBCSWE01a ~]# sysctl net.ipv4.tcp_window_scaling
1
[root@SBCSWE01a ~]# sysctl -w net.ipv4.tcp_window_scaling=0
[root@SBCSWE01b ~]# sysctl net.ipv4.tcp_window_scaling
1
[root@SBCSWE01b ~]# sysctl -w net.ipv4.tcp_window_scaling=0

To be persistent across the SBC restart and system reboot, use the Linux the following shell command on both active and standby.
[root@SBCSWE01a ~]# echo "net.ipv4.tcp_window_scaling=0" >> /etc/sysctl.conf
[root@SBCSWE01b ~]# echo "net.ipv4.tcp_window_scaling=0" >> /etc/sysctl.conf

There was a switchover with the PRS process coredumping.

Impact: The PRS process coredumped due to memory corruption.

Root Cause: Customer encountered a PRS process core due to the memory corruption that happened earlier in processing. The core is the after effect of earlier corruption.

Steps to Replicate: None.

The code is modified to avoid the core, and to collect more diagnostic data if a core happens again.

Workaround: N/A

Portfix SBX-100779 (Originated in Release 7.2.5): The SBC was adding an extra application/ISUP ACM content.

Impact: The 200 OK Bye has duplicated the ISUP body.

Root Cause: When the e2e Bye is enabled, 200 OK has duplicated the ISUP body.

Steps to Replicate:

1. Configure the isupMimeBodyRelay and e2e bye.
2. Run a sipi-sipi call, and the SBC sends 200 OK bye has duplicated isup body.

The code is modified so the SBC does not create an isup body internally for the e2e Bye and isupMimeBodyRelay.

Workaround: Disable the e2e Bye.

There was a back-to-back coredump on the Server.

Impact: The SCM process cored due to memory corruption while parsing an INVITE with a very long and invalid CallId field, while the parseError trace logging was enabled.

Root Cause: The SCM process coredump is caused by memory corruption that is the result of the Trace Logging code overwriting a buffer.

This can happen when the SIP parser encounters an invalid CallId that is also very long.
It is also possible that other parsing errors can trigger this core, especially if the parse error was encountered for a particular header that is very long.

Steps to Replicate: This can be reproduced by enabling the parseError trace logging and then sending a INVITE that includes a badly formatted CallId, which is the maximum allowed size for a CallId:
"set global callTrace errorFilter errorType parseError"

The code is modified to not overwrite the end of the buffer when it is attempting to write a very long error message string.

Workaround: Disable the parseError tracing.

A BYE message was sent to the wrong port.

Impact: The SBC does not update remote connection address when refresh INVITE is received from different port.

Root Cause: When the endpoint is registered over TLS initiating a call and after the call is established, the endpoint sends a refresh register from a modified port. When a refresh INVITE is sent with no SDP change, the SBC does not send BYE to a modified port when the call disconnects.

Steps to Replicate: 

1. Register endpoint over TLS.
2. Endpoint initiates a call, call gets connected.
3. Endpoint sends refresh register from modified port.
4. Endpoint sends refresh re-INVITE from modified port.
5. Called party disconnects the call.
6. Verify that the SBC sends BYE to modified port towards caller.

The code is modified to ensure the SBC updates the remote address when refresh INVITE is received from modified port. This ensures when the registration is required and for a TLS connection, after the call is released BYE is sent to correct port.

Workaround: N/A

The SMM for changing ISUP message inside SIP-I does not work properly.

Impact: When a message is received with mime content having single msgbody, then after the SMM application on the msg body, the SBC re-formats it as a normal msgbody instead on the mime content. The re-formating was not handled properly.

Root Cause: When a message is received with mime content having single msgbody, then after the SMM application on the msg body, the SBC re-formats it as normal msgbody instead on mime content. 

Steps to Replicate: 

1. Add the SMM rules that applies on the message body.
2. Send message with mime content having single msgbody in it.

The code is modified so that whenever the SBC re-formats the mime body after the SMM application it formats correctly.

Workaround: N/A

Portfix SBX-101294 (Originated in Release 7.2.5): Intermittently, the SBC SWe accesses the sipSigPort for sending the SIP messages to the core side.

Impact: The relay options with registration may route to the wrong Sip Signaling Port and causing call fails from AS to IAD.

Root Cause: When the IAD sends a registration from SSP1 and relay options from an other SSP2. Since the options from the SSP2 do not match with RCB from SSP1, the SBC treats the options from the AS, triggering an invalid routing to the wrong destination.

Steps to Replicate: Configure the SSP1 and SSP2, and enable maskRcbPort. The registration from SSP1 succeeds. Options from the SSP2 will trigger a relay back to the IAD (not AS). Subsequent calls from AS may fail or refresh registration.

The code is modified to detect the relay messages from the IAD or AS based on RCB looks up (not just based on SSP).

Workaround: Disable the maskRCBPort. Or use the SMM to make the SBC respond to OPTIONs (not relay).

Portfix SBX-100479/480: Observed DSP and MAJOR logs coredumps on the 7000 Platform while running the ICM Scaling suite.

Impact: The DSP and MAJOR logs coredumps on the 7000 Platform.

Root Cause: Certain variables used in the interrupt context were not protected while being modified outside the interrupt. This impacted the inter-core health-check logic at the DSP and incorrectly led to an error being reported, eventually leading to a coredump.

Steps to Replicate: From an HA Pair of fully loaded SBC-5100/5200/5110/5210/5400/7000

1. Run a transcode load.
2. Initiate a switchover.

The code is modified so variables are now interrupt-protected.

Workaround: N/A

Portfix SBX-101547 (Originated in Release 8.2.2): There was a switchover and coredump after call placed on hold.

Impact: The code was dereferencing a null pointer and causing the SCM process to crash.

Root Cause: The customer had the passThruPrivacyInfo control enabled on both the ingress and egress privacy profiles. But the ingress side of the call did not pass through some expected information related to the INVITE message and it caused the code to dereference a null pointer.

Steps to Replicate: Unable to reproduce this issue, the code is fixed based on code review and coredump analysis.

The code is modified to validate the pointer is not null before trying to use it.

Workaround: Disable the passThruPrivacyInfo controls.

The call is failing intermediately due to the Unsolicited Call Cleanup.

Impact: The call is failing intermediately due to the Unsolicited Call Cleanup

Root Cause: The affected call failed due to the XRM having received an error response from the NP for modify RID command, which was most likely the race condition in the NP. This issue occurred because the RID enable and modify commands were issued within the same seconds but from two different NP users.

Steps to Replicate: Due to the nature of race condition, it is not guaranteed to reproduce the problem.

The code is fixed based on a code review.

The code is modified to use the same NP interface user handle when sending the RID enable and modify commands to the NP for that special case.

Workaround: N/A

Portfix SBX-99579 (Originated in Release 9.0.0): Observed the Ema process core dump on the S-SBC during a sbxrestart. 

Impact: The EMA process may coredump and the application may not come up after an upgrade if the SWE/Cloud has a cinder volume attached for log storage.

Root Cause: While upgrading the SWE/Cloud platform attached with cinder volume, file ownership and permissions in the cinder volume are untouched causing wrong set of ownership on the EMA log file resulting in the EMA failing to start.

Steps to Replicate: Perform an upgrade from 7.x to 8.2.3R0 in cloud with cinder volume and check the sbxstatus and check for coredumps. With the problematic build, the EMA coredumps can be observed.

The code is modified so during an upgrade, the EMA log dir ownership is set to the sonusadmin:sonus to ensure the EMA process can access the log file and can start without issues.

Workaround: Manually change the ownership of the EMA log dir after an upgrade with below command:
chown -R sonusadmin:sonus /var/log/sonus/ema/

Portfix SBX-102006 (Originated in Release 7.2.5): The SBC failed to send ACK to the right IP.

Impact: After a call was connected with egress received RR in 2xx, the SBC sends a reInvite, and received the rexmit of 2xx. Later on, the SBC sends ACK to the wrong destination.

Root Cause: A rexmit of 200OK, accidentally deletes the previous RR.

Steps to Replicate:

1. A calls B.
2. B responds with a 200OK with RR.
3. A reInvite triggers the SBC reInvite to B.
4. B responds with 2xx and rexmit 2xx.
5. The SBC sends ACK to the wrong destination (based on contact not from previous RR received).

The code is modified so the SBC ignores updating the data of the dialog.

Workaround: N/A

Portfix SBX-99020 (Originated in Release 9.0.0): Observed Major logs with the error prints of 488 on the SBC 5210 platform while running direct media.

Impact: The calls stop working as LIF bandwidth is not freed.

Root Cause: In the case of a multi stream direct media call with the last media stream as non-RTP stream, the SBC does not release the session bandwidth even when the call is released. This leads to a leakage in interface bandwidth resulting in call failures after some time.

Steps to Replicate: Run direct media calls with multiple streams with the last media stream as a non-RTP stream (BFCP, MSRP).

The code is modified to release the session bandwidth even when the last media stream is non RTP stream in a direct media call.

Workaround: N/A

The call forking with the MS Teams and GW-GW fails.

Impact: The call forking with a GW-GW to might not work correctly.

Root Cause: The call forking logic was not copying all of the internal parameter information when grouping the routes based on each AoR tag. This meant the SRTP enabled information was lost and the call was initiated using plain RTP instead of the SRTP.

Steps to Replicate: Make a call forking call over GW-GW where one egress end point requires SRTP and the other does not. Check that the SRTP call is correctly set up.

The code is modified to correctly copy all of the route parameters between the PSX response and the forking queues.

Workaround: N/A