In this section:

Note

Only an Administrator and a Network Operator can access the Troubleshooting setting (enabling/disabling). However, the Guest User can view the CDR Viewer page, and if enabled by the Administrator/Network Operator, the Guest User can perform all the operations on the CDR Viewer page. If this option is disabled, the Guest User cannot enable Troubleshooting settings.

The CDR Viewer page is restricted for other User Groups like Security Auditor and Calea Users.

EMA UI Menu

On the SBC main screen, navigate to Troubleshooting > Troubleshooting Tools > CDR Viewer.

The following figure represents the CDR Viewer screen:

The following CDR fields are available:

    • Record Type
    • Record Date/Time
    • Calling Number
    • Called Number
    • GCID
    • Call Disconnect Reason
    • Sip Ladder
    • Call Duration 

The user can capture and view the CDR event when polling is enabled. Refer to Dashboard - Live Monitor for more information on polling.

Enabling or Disabling CDR Data Polling

  1. On the SBC main screen, navigate to Troubleshooting > Troubleshooting Tools > CDR Viewer.
     

  2. To enable/disable CDR Data Polling, click the desired option as shown in the figure Enable-Disable CDR Data Polling. By default, it is Disabled for the Troubleshooting page. The CDR records are fetched from ACT records, which are generated by the SBC. If Polling is enabled, the CDR records are captured and stored in the Database. The following figure represents Enable-Disable CDR Data Polling in the CDR Viewer screen:

    Note

    Enabling CDR Viewer on SBC SWe with less than 6 CPUs and less than 16 GB RAM may impact the system performance.

Filtering the CDR List

Filter the CDR list based on the following fields:

  • Record Type
  • Record Start Date/Time range
  • Call Duration
  • Calling Number
  • Called Number
  • GCID
  • Call Disconnect Reason

The following figure represents CDR Filters:

Viewing CDR Records

Select any CDR record to view the details. The following figure represents CDR details:



Search for Specific Text

Users can also search (case-insensitive) for a specific text within the record details of a selected CDR record. The following figure represents the CDR search using Find:

CDR Data Purge

CDR data is purged either on a Disk or Age basis. The Disk-based purging occurs when the database reaches 20 percent of the evlog partition. The Age-based purging occurs for data older than one week. The user cannot view CDR records (including CDR details) and the SIP Ladder Diagram that are older than one week.

The Disk-based purging always takes precedence over Age-based purging. All users have the ability to view the CDR Viewer page, but only the Administrator and the Operators have the ability to change the settings for the CDR Viewer page.

Note

The system sets the purge limits, and they are not user-configurable.

SIP Ladder

Note
  • SIP Ladder diagrams are generated for calls captured through Level 4 Call Trace Filter. The Level 4 filters are based only on the Peer IP Address, and it must be specified to trace calls. For more information on Level 4 Call Trace Filter, refer to Call Trace and Packet Capture - Call Trace.
  • SIP Ladder diagrams can be captured at up to 50 CPS.
  • No other trace filters can be running at the same time as the level 4 trace (that is, do not run a level 1, 2, or 3 trace at the same time as the level 4 trace.)
  • Level 4 filters identify GCID when possible. However, some messages do not contain GCIDs and may have 0xfffffffff as the GCID (registration, notify, options ping, and so on).
Note

Message Length Limitation: The SIP Ladder diagram tool displays call messages by parsing message entries recorded in the trace (.TRC) log file. The TRC log has a message-size limit of 1800 bytes per entry, including a header which includes a time-stamp and sequence number. The header normally contributes fewer than 100 bytes. If a message within the call exceeds the 1800-byte limit, the TRC log divides it into two or more blocks and adds a header to each block. The SIP Ladder diagram cannot display the subsequent blocks of a message if it is divided into multiple blocks.

  • When the SIP Ladder is enabled, the EMA only supports up to 50 CPS with 14 messages per call or 700 SIP messages per second. If the number of messages in the TRC file exceeds the limit of 700, the SIP ladder viewer does not work correctly. For example, it can lead to slow load time, and node congestion in the SBC which in turn triggers alarms generation and drop in traffic.
  • To determine the number of messages per second, the user can count the number of messages in the TRC log per second.

Enabling or Disabling the SIP Ladder Tool

  1. On the SBC main screen, navigate to Troubleshooting > Troubleshooting Tools > CDR Viewer.  

  2. To enable/disable SIP Ladder, click the desired option as shown in the figure Enable-Disable SIP Ladder. By default, it is Disabled for the Troubleshooting page. The CDR records are fetched from ACT records, which are generated by the SBC. If Polling is enabled, the CDR records are captured and stored in the Database. The following figure represents the Enable-Disable SIP Ladder in the CDR Viewer screen:

SIP Ladder Diagram 

The SIP Call Ladder Diagram is a tool used to identify faults in calls. Calls are analyzed at a higher level first, and then at lower levels within the components using the SIP Ladder Diagram. To view the SIP Ladder diagram, click the SIP Ladder icon in the Sip Ladder column for a CDR entry in the CDR Call List. The following figure represents SIP Ladder diagram icons:


A SIP Call Ladder Diagram provides the following advantages:

  • Viewing the high level signaling path that includes the Originator, Intermediate Destination and Final Destination of calls to know the complete call path.
  • Viewing the SIP Protocol Data Unit (PDU) details either by hovering the mouse on message types or by clicking the message types (INVITE, PRACK, 200 OK, and so on) to further analyze the calls. Users can search within the PDU pop-up window.
  • Viewing the signaling call ladder diagram to narrow down any issues in the call.
  • Performing a drill down analysis of a failed call by further extracting the log details that occurred around that instance. These log details are viewed either on a pop-up window for a quick analysis or downloaded as call logs (.TRC logs) for that instance.
  • Viewing Error Messages, Possible Root Causes and Recommendations. 

A SIP Ladder diagram is not displayed if the Call Trace is not running and if the SBC is not processing any calls for a minimum duration of two seconds.

Click the options available in a SIP Ladder Diagram to:

  • View the exact timestamp next to each message (for example, INVITE, 200 OK, ACK and so on) in the SIP Ladder Diagram.
  • View the IP Address and the Port for each leg of the call.
  • View the request type for each message in the call.
  • View and search the SIP PDU for each message in the call. The SIP PDU is shown as a tooltip when you hover the mouse pointer over the horizontal lines of the SIP Ladder diagram.
  • View multiple SIP PDUs on the SIP ladder diagram at the same time.
  • View the log information as a pop-up message when you click the horizontal lines in the SIP Ladder diagram. This log information is downloaded by clicking the Download button in the SIP Message detail pop-up window.
  • View and search the log details from the TRC, DBG, and SYS logs for an individual message.
    • The log details will only be displayed for log lines that contain the GCID of the selected call within the selected time period.
    • For the first message, log details are displayed for the time between the message itself and the next message.
    • For the last message, log details are displayed for the time between the previous message and the message itself.
    • For all other messages, log details are displayed for the time between the previous message and the next.
  • Download the log details from the TRC, DBG, and SYS logs for an individual message to a file.
    • The log details will only be displayed all log lines within the selected time period.
    • For the first message, log details are displayed for the time between the message itself and the next message.
    • For the last message, log details are displayed for the time between the previous message and the message itself.
    • For all other messages, log details are displayed for the time between the previous message and the next.
  • Downloaded log contains all log lines. The complete log data can be sent to Ribbon for evaluation if required.

  • Display a tooltip with the details of each SIP Message when hover over the horizontal line below the respective SIP message in the SIP Ladder diagram. The following figure represents SIPing Ladder Tooltip:

     


  • The user can download all the messages or the complete SIP PDU, which belong to the same call as presented in the SIP Ladder Diagram in a single file. The SBC provides the following information in the complete SIP PDU message:

    • CDRs at the beginning of the file
    • Directions of the messages (ingress/egress or received/sent)
    • Transport protocol (UDP/TCP/TLS/SCTP)
    • Date next to the time (part of timestamp)
    • IP addresses with the port numbers of the senders and receivers

    In the SIP Ladder Diagram, Select Download All to download all the messages in a single file. A pop-up window is displayed to download the complete SIP PDU. The following figure represents downloading the complete SIP PDU:


    By default, the complete SIP PDU message opens in .txt format. The following figure represents the Complete SIP PDU message:



  • Displays a pop-up window with the SIP PDU when you click the SIP Message. SIP PDU can be downloaded as a .text file by clicking the Download button within the pop-up window. The SIP PDU pop-up window contains a search box, which is used to search and highlight the text present in the pop-up window. The following figure represents the SIP PDU pop-up window:

    The following figure represents the SIP PDU pop-up window showing the search box:


  • Displays the log information as a pop-up message when you click the horizontal lines in the SIP Ladder diagram. This log information is downloaded by clicking the Download button in the SIP Message detail pop-up window. The log information pop-up window contains a search box, which is used to search and highlight the text present in the pop-up window.

    The following figure represents log information:
     

    The following figure represents the log information pop-up window showing the search box:

  

SIP Ladder Performance Tests

The following table lists SIP Ladder performance test results on the SBC ERE based on the following conditions.

  • The data in the table below was generated from tests performed by the Ribbon internal testing organization.
  • Testing was done using the SIPP call simulator with simple calls.
  • The tests measured CPU utilization, memory utilization and call failures. There was no system degradation for each test condition, and call failures were well within the acceptable limit.
  • The SIP ladder diagram was verified to ensure that it was generated correctly under different test conditions.
  • As of release 5.0,  call processing is separated from SIP ladder processing by allocating dedicated CPUs for each process. Exceeding the call trace 50 cps or 450 PDUs/second limit may affect SIP ladder processing and thus delay generating the SIP ladder diagram.
Note

The SBC logs store approximately 1200 EVENTS per second combined for CDR viewer and SIP ladder functionality. These EVENTS include CDR START, STOP, INTERMEDIATE, and ATTEMPT records from the CDR viewer and all the SIP Requests and response messages from the SIP Ladder Diagram. 

For example, on the SBC 5400 platform, when the traffic is running @ 400 cps, the number of CDR EVENTs generated per second = 800 (400 START + 400 STOP records).

Out of the 1200 supported EVENTS, the SBC receives 800 EVENTS per second for the CDR viewer. Thus, the SBC may log another 400 EVENTS per second for the SIP Ladder Diagram, which includes all the SIP requests and responses.

The basic SIP call contains seven EVENTS (INVITE, 100 Trying, 180, 200, ACK, BYE, 200). With the remaining 400 EVENTS logging available for the SIP Ladder Diagram, the SBC supports approximately 50 simultaneous calls per second.

Key:

E - Enabled

D - Disabled

Load Duration/SPAM values are updated once the load is in use. The following table represents the SIP Ladder performance test results:

SBC Platform   

RAM
(GB)

Live Monitor

Troubleshooting

SIP Ladder

CPS

CHT
(seconds)

Load Duration
(hours) 

540032EEE

435
(SIP Ladder - 35)

10048
7000128EEE200
(SIP Ladder - 50)		3012
7000128DEE950
(SIP Ladder - 50)		3060
7000128EED135011018
SWe (4vCPU)14EEE30
(SIP Ladder - 15)		4573
SWe (8vCPU)14EEE70
(SIP Ladder - 40)		2677
The following table represents column descriptions:
SBC PlatformPlatform on which performance tests were done.
RAMTotal RAM available in the SBC.
Live MonitorIndicates whether Live Monitor feature was enabled (E) or disabled (D).
TroubleshootingIndicates whether CDR Viewer feature was enabled (E) or disabled (D).
SIP LadderIndicates whether SIP Ladder feature was enabled (E) or disabled (D).
CPSIndicates Calls Per seconds used. The number in the bracket indicates the calls for which SIP ladder was enabled. For example, 150 (SIP Ladder - 50) indicates 150 calls per second, out of which 50 calls were being traced for the SIP ladder diagram. i.e. For 100 calls there were no matching call filter and for 50 of them there were matching call filter and SIP ladder was displayed for these 50 calls.
CHT (seconds)Indicates call hold time in seconds of each call.
Load DurationIndicates duration in hours for which the calls at the specified cps were run.

How to Increase CDR Repopulation Speed Post-Upgrade

After an upgrade, CDRs can take a lengthy amount of time (60+ minutes) to repopulate. Please note the following to resolve the issue:

  • Repopulating CDRs is a CPU-intensive task. The CDR viewer and SIP ladder are not able to present all of the old information until the old ACT/TRC files are reprocessed and stored internally.
  • Therefore, if the user does not require the old ACT/TRC files, then the user can delete those files prior to re-enabling the troubleshooting tools so there is less data to re-import. Deleting the ACT/TRC files will significantly increase CDR repopulation speed.
  • The user must re-enable the troubleshooting tools post-upgrade. Also, the user is advised to re-enable the troubleshooting tools only during a maintenance window.