In this section:

Use the Configure Trace and Media Packet Capture window to both configure Call Trace Filters and to initiate call traces to use in debugging policy and call routing issues. 
 

On the SBC main screen, go to Troubleshooting > Call Trace/Logs/Monitors > Call Trace and Packet Capture > Call Trace. The Configure Trace and Media Packet Capture window opens. 

Configure Trace and Media Packet Capture Window

 

There are two panels in this window: the Call Trace Status and Setting panel and the Call Trace Filters panel. The Call Trace Filters panel lists any existing Call Trace Filters.

Call Trace Filters define conditions that determine which call events trigger logging PES call trace messages to the system trace data event log. When a call meets the criteria, the trace information is written to a tracing file (.TRC). The following default information is collected during a call trace: 

  • Service group configuration
  • DSP T.38 event information, if applicable
  • DSP channel statistics
  • DEBUG/SYS log information  
Warning

Do not configure a Call Trace Filter to trace all calls. Use of the Call Trace feature can have significant impact on the network performance of the SBC system. Ribbon strongly recommends configuring Call Filters which result in a small number of calls being traced to avoid heavy loads. Also, be sure to delete Call Trace Filters once you have obtained the necessary information.

Call Trace Filters

Note:

Only Administrators, Field Service, Guests and Operators have the ability to access the Call Trace configuration screen.

You must create Call Trace Filters before you can start a call trace.

 The Call Trace Filter object allows you to create and configure Call Trace Filters that specify criteria that the SBC uses to determine which call events to trace.

To Create a Call Trace Filter

To create a new Call Trace Filter, click New Call Filter on the Call Trace Filters panel. The Create New Call Trace Filter window opens.

Create New Call Trace Filter Window

 

 

Enter values for the Call Trace Filter object using the following table and click Save.

Call Trace Filter parameters

 

Parameter

Description

NameSpecifies the name of the Call Trace Filter of up to 23 characters.
State

Specifies the state of this filter. The options are:

  • Disabled (default): If disabled, it is not used for a Call Trace.
  • Enabled: If enabled, this filter can be used for a Call Trace.
SIPRec Legs Capture

Enable this flag to trace and capture packets for SIPREC legs.

  • Disabled (default)
  • Enabled

For more information on this flag, refer to Call Trace to Capture SIPRec Legs.

Capture type

Specifies the type of logs to capture while running a Call Trace. The options are:

  • Capture trace information (.trc logs) only
  • Capture trace information (.trc logs) and media information (.pkt logs).
Detail level to log

Specifies the level of details that need to be capture in the log file when the trace is complete. The options are:

  • Level 1 - Everything
  • Level 2 - Everything but raw hex dump
  • Level 3 - Only external message information and errors
  • Level 4 - Only SIP PDU events 

Note: The Level 4 log is implemented to capture SIP Ladder diagrams for the interpretation of calls. Level 4 Call Trace Filter conditions/behavior:

  • Level 4 traces all the SIP PDU events (without logging other trace messages) at line-rate without dropping any PDUs. These traces are used by SIP Ladder feature for further analysis.
  • The Level 4 filters are based only on Peer IP address and this must be specified to trace calls. This call trace method does not trace/filter based on trunk group, called number, calling number, and so on.
  • Configure a Level 4 filter with a wildcard Peer IP address  of 255.255.255.255 to match ALL packets to/from any IP address.
  • Level 4 filters identify GCID when possible. However, some messages do not contain GCIDs and may have 0xfffffffff as the GCID (registration, notify, options ping, and so on).
  •  Other filters may run simultaneously as trace level 4, but ensure configuring the parameter maxTriggerCount to a non-zero value by executing the following command:  
    set global callTrace maxTriggerCount <1-64>  
    where the allowed range for maxTriggerCount is 0-64. For more information, refer to Call Trace - CLI.  

For more information on the effect of different levels with respect to the flag SIPRec Legs Capture, refer to Call Trace to Capture SIPRec Legs.

Capture calls that match these filters

Specifies the criteria to use determine which calls to capture in the logs in a Call Trace (these are optional). The options are:

  • Called number: Specifies the called party number to trace. Field length is up to 30 characters.
  • Calling number: Specifies the calling party number to trace. Field length is up to 30 characters.
  • Contractor number: Specifies the contractor number to trace. Field length is up to 30 characters.
  • Redirecting number: Specifies the redirecting number to trace. Field length is up to 30 characters.
  • CDDN number: Specifies the Called Directory Number to trace. Field length is up to 30 characters. 
    Note: For the preceding options, the following characters are accepted by the SBC when specifying Call Trace Filter criteria: 

    • Digits 0-9

    • Plus (+) symbol

    • Wildcard matching is also supported. Use "X" or "x" wildcard character to match any single digit, and "%" symbol to match any digits from that point forward.
      For example, using "978xxx1212" returns all calls between 9780001212 and 9789991212, and using "978%" returns all calls with a 978 prefix. 

  • Transfer capability: Specifies the transfer capability of the call trace filter. Options are: 
  • Audio31Khz - 3.1 kHz Audio - ITC 3.1 kHz audio calls are traced. The default setting is disabled.
  • Speech - ITC Speech based calls are traced. The default setting is disabled.
  • Unrestricted - Calls with ITC Unrestricted are traced. The default setting is disabled.
  • Unrestricted With Tones - Unrestricted Digital Information With Tones Announcements - Calls with ITC Unrestricted Digital Information With Tones Announcements are traced. The default setting is disabled.
  • Video - Calls with ITC Video are traced. The default setting is disabled.
  • Trunk Group: The name of a trunk group to filter the calls to trace. This string must match the trunk group name exactly. You cannot use wildcard characters for this argument. Field length is up to 23 characters.
  • Peer IP address: Specifies the peer IPv4 or IPv6 address for call tracing.
Stop Match

Specifies the condition to stop matching filters. The options are:

  • When a match occurs in this filter, stop trying to match the other filters
  • Continue to try to match up to two other filters after a match is found in this filter (default)
Note

As a general rule, set the Call Trace Filter for the ingress side except for the Called number. 

To Edit a Call Trace Filter

  1. Select an existing Call Trace Filter object by clicking the radio button adjacent to it in the Call Trace Filters window. The Edit Call Trace Filter window is displayed.

  2. Make changes and click Save.

To Copy a Call Trace Filter

  1. Select an existing Call Trace Filter object by clicking the radio button adjacent to it in the Call Trace Filters window.
  2. Click Copy Call Filter. The Copy Call Trace Filter window is displayed.
  3. Make changes and click Save. 

To Delete a Call Trace Filter

  1. Select an existing Call Trace Filter object by clicking the radio button adjacent to it in the Call Trace Filters window.
  2. Click the X on the right end of the selected row.

  3. Confirm the deletion when prompted.

Call Trace Status and Settings

Once you have one or more Call Trace Filters created, call traces can be started and stopped from the top panel of the Configure Trace and Media Packet Capture window. See the earlier figure of the Configure Trace and Media Packet Capture window. There you can start, save and stop traces using the enabled filters. The trace continues running until a specified duration elapses or the specified number of call filter matches is reached.

Note:

When you start a trace, by default, the first three enabled filters in the table are used for the trace.

To Run a Call Trace

 In the Call Trace Status and Settings panel, choose the required conditions for the trace using the following table.

Call Trace Status and Settings parameters

 

Parameter

Description

Call Trace Duration

Specifies the duration for the Call Trace to run. The options are:

  • Run trace until stopped (by clicking the Stop Trace button): If selected, the trace will keep running till it is stopped manually (by clicking the Stop Trace button).
  • Stop trace after __ minutes: If selected, add the number of minutes for the Call Trace to run. The value ranges from 1 to 360 minutes and the default value is 180 minutes.
Number of Matches (optional)Specifies the maximum number of calls to match before stopping the trace. The value ranges from 1 to 64 times.
Trace Calls with errors of the type

Check this object to enable specifying  a system-wide call trace error filter trigger criteria. By default the option is unchecked which means there is no SIP PDU logging for this error filter. After checking the option the following options are available:  

  • Any—Log all SIP PDUs that meet the requirement of the Parse Error, Out of Dialog, and Early Attempt options below.
  • Parse Error—Log received SIP PDUs that fail parsing. If an error is detected by SIP signaling, the accompanying PDUs will not be associated with any dialog, including an existing dialog. Hence all in dialog and out of dialog PDUs are assigned to this category. If an error is detected at the front end pre-parser, the PDU is logged by this software and not forwarded to SIP signaling. This prevents an offending PDU from being logged by both the pre-parser and SIP signaling.
  • Out of Dialog—Log incoming syntactically correct SIP PDUs (except INVITE) that do not belong to an existing dialog.
  • Early Attempt—Logs incoming SIP PDUs that result in early termination of the call (by either the pre-parser or SIP signaling).

Note

The system-wide error filter triggers a call trace based on the selected type of error. If you select to trace calls with errors as well as define call filters to trace, the error filter runs in parallel with the call filters. Both may trigger producing a double-PDU output, one for the conditions matched in the call filter, and one for the detected error.

For further details on error logs, see the section Call Trace Log Messages.

Call Trace Status

Displays the current status of the Call Trace.

  • When there is no trace running, the status is Call trace stopped with a red dot.
  • when a trace is running, the status is Call trace running with a green dot.

Click Save & Start Trace to start the trace. The trace status is displayed in the same panel.  

Call Trace Logs and Monitors - Call Trace Running

The trace stops automatically if a time limit is specified, otherwise you must stop the trace manually by clicking Stop Trace.

Call Trace Logs and Monitors - Call Trace Stop

Once the trace is complete, the logs are generated and stored at Log Management. Look for TRC and PKT log files (.trc and .pkt log files). For more details on log messages, see the section Call Trace Log Messages.

Call Trace Log Messages

The memory available for Call Trace Log messages are limited to 1800 bytes per log entry. If a log entry occupies memory more than the 1800 bytes allocated for it, the entry is divided into two entries. The next 1800 bytes are allocated for the new log entry with the same time-stamp and sequence number. This ensures that a human or a program can parse the entries, and if necessary, can reassemble the entries with the same time-stamp and sequence number into one single chunk.

A sample log entry with memory footprint of more than 1800 bytes is illustrated below (observe the comments within /* */):

A sample log entry with memory footprint of more than 1800 bytes

242 12162016 083615.736943:1.02.00.38938.Info .SIPSG:CALL:FILTER=Cisco_Test_Call GCID=0x00000DDB: 14443. TraceCallLogger invoked - From pstProperties: PeerIP/port: 10.100.100.10/5000 LocalIP/port:10.139.160.141/5000 ulMsgType-0x40000004

/* This is a comment for illustration and not part of the actual log entry produced by the SBC. */
/* The line just below this comment indicates the start of an entry in the log file. */

1787 12162016 083615.737177:1.01.00.38939.MAJOR .SIPSG:FILTER=Cisco_Test_Call GCID=0x00000DDB: EXTRA INFO: Ingress: received msg for CallId:804ee913ccfe61c2aa581a72a000 gcid:0xDDB from IP/port:10.100.100.10/5000, Local IP/port:10.139.160.141/5000
RAW PDU:
INVITE sip:+8005555555@abc.company.com:5000 SIP/2.0
From: "AU-User-A" <sip:+8005555556@fgh.company.com>;tag=2ea08b0a-13c4-5853a77f-e2852bca-687922
To: <sip:+8005555555@ijk.company.com>
Call-ID: 804ee913ccfe61c2aa581a72a000
CSeq: 1 INVITE
Via: SIP/2.0/TCP 10.100.100.10:5000;branch=z9hG4bK-1fe0f0-5853a77f-e2852bca-53b01fb3
P-AV-Message-Id: 1_1
History-Info: <sip:+8005555555@lmn.company.com>;index=1, "8005555555" <sip:+8005555555@lmn.company.com>;index=1.1
P-Asserted-Identity: "AU-User-A" <sip:+8005555556@fgh.company.com>
Max-Breadth: 60
P-Charging-Vector: icid-value="AAS:409-910e4e801e6cf3c1a58aac0a072"
Alert-Info: <cid:internal@opq.company.com>;avaya-cm-alert-type=internal
Accept-Language: en
Supported: 100rel,histinfo,join,replaces,sdp-anat,timer
User-Agent: Avaya CM/R016x.03.0.124.0 AVAYA-SM-6.3.18.0.631804
Max-Forwards: 66
Av-Global-Session-ID: 804e0e91-3ccf-4601-b7aa-581a72a00000
P-Location: SM;origlocname="ACM4 R3 AU";origsiglocname="ACM4 R3 AU";origmedialocname="ACM4 R3 AU";termlocname="T2A R3 AU";termsiglocname="T2A R3 AU";smaccounting="true"
Session-Expires: 1800;refresher=uac
Min-SE: 1800
Allow: INVITE,ACK,OPTIONS,BYE,CANCEL,SUBSCRIBE,NOTIFY,REFER,INFO,PRACK,PUBLISH,UPDATE
Contact: <sip:+8005555556@10.100.100.10:5000;transport=tcp>
Content-Type: application/sdp
Content-Length: 253
 
v=0
o=- 8420266 0 IN IP4 10.240.42.55
s=-
c=IN IP4 10.240.42.111
b=AS:64
t=0 0
a=avf:avc=n prio=n
a=csup:avf-v0
m=audio 49080 RTP/AVP 18 127
a=rtpmap:18 G729/8000
a=rtpmap

/* This is a comment for illustration and not part of the actual log entry produced by the SBC. */
/* The line just after this comment indicates the start of a new entry in the log file, but with the same sequence number and time-stamp. */
/* It indicates that the new entry is an extension of the previous entry, after the quota of 1800 bytes of memory was utilized by the previous entry. */

148 12162016 083615.737177:1.01.00.38939.MAJOR .SIPSG:FILTER=Cisco_Test_Call :127 telephone-event/8000
a=fmtp:18 annexb=no
a=ptime:20
a=sendrecv
 

 

Messages like INVITE, in combination with SIP Headers like P-Asserted-Identity, P-Location, etc. often tends to produce log entries with memory footprint of more than 1800 bytes per entry.

Additional information about a call is indicated by the EXTRA INFO delimiter (see the example above). Currently, it comprises of the following:

  • Ingress/Egress indication in text
  • Indication if the message is received or sent
  • CallId
  • Global Call ID (gcid)
  • from IP/Port, to indicate the source IP and port of the message
  • LOCAL IP/Port, to indicate the recipient IP and port of the message
  • RAW PDU, which is the raw PDU in ascii text but not encrypted. The RAW PDU section is the largest in a log entry, and often wraps between log entries. Sometimes, it even spans over multiple files.

Offline Call Trace Analysis Using LX Utility

The TRC files can be analyzed offline using the LX tool, which generates a SIP ladder diagram from the TRC or DBG files. The LX tool is a Windows PC application available free of charge to all customers and partners with a license agreement, and can be downloaded as part of your software download request. For details, refer to LXDOC.

LX Main User Interface