Call Trace and Packet Capture - Call Trace

In this section:

Call Trace

The SBC supports configuring Call Trace Filters using the Signaling Packet Capture command to establish criteria to determine call events and logging PES call trace messages (using the ".TRC" file extension) to the system trace data event log. This information is used for debugging policy and call routing issues.

The following default information is collected during a call trace:

Service group configuration
DSP T.38 event information, if applicable
DSP channel statistics
DEBUG/SYS log information

The first three Call Trace filters are automatically logged unless the stopMatch flag in enabled, whereby the number of filters logged may be less than three. The stopMatch flag will halt filter profile matching once a match is found.

When call trace filter criteria is provisioned and a call meets the criteria, trace information is written to a tracing file. Call trace is enabled using the provisioning interface.A callTraceTimer configuration is provided along with the maxTriggerCount configuration to limit the logging of call/error filter events in the TRC file. The SBC may be configured to stop call tracing if either of following criteria is met:

Configured maxTriggerCount is reached or,
Configured callTraceTimer expires

When a call trace filter is created, call trace is started, and a call fulfills the filter criteria, the trace information is written to a tracing file (.TRC). Call trace can be started and stopped from the Call Trace > Configure Trace and Media Packet Capture screen. The trace continues running until the specified duration or number of call filter matches is reached.

The EMA Call Trace feature supports the following functionality:

Configure Call Trace, Call Filter, Match, and Error Filter options from a single screen.
View the status of a Call Trace.
Stop a Call Trace with a single command.
Run the Call Trace for unlimited duration.
Restart the Call Trace with a single click when a call trace gets stopped once the call filters reaches the threshold limit.
Delete any Call Filter.
Create and Edit Call Filter and Filter Match Criteria from a Single Tool.

Call Trace Filters

Warning

Do not turn on the Call Trace Filter to trace all calls. Use of the Call Trace feature can have significant impact on the network performance of the SBC system. Sonus strongly recommends configuring Call Filters which result in small quantities of calls being traced to avoid heavy loads. Also, be sure to delete Call Trace Filters once you have obtained the necessary information.

Note:

Only the Administrators, Field Service, Guests and Operators have the ability to access the Call Trace Configuration screen.

The Call Trace Filter object allows you to create and configure Call Trace Filters that specify criteria that the SBC uses to determine call events added to the system trace data event log (which uses the .TRC file extension).

Note

As a general rule, set Call Trace to ingress mode unless egress mode is mandatory.

Navigation Path:

On the SBC main screen, navigate to Troubleshooting > Call Trace/Logs/Monitors > Call Trace and Packet Capture > Call Trace. The Configure Trace and Media Packet Capture window appears.

Call Trace Logs and Monitors - Call Trace Main Window

There are two panels for this window: Call Trace Status and Setting panel and Call Trace Filters panel. The second panel reflects the Filter entries in it.

You need to create Filters first or use the created Filters to start a Call Trace on it.

To Create a Call Trace Filter

To create a new Call Filter, click New Call Filter tab on the Call Trace Filters panel. The Create New Call Trace Filter window is displayed.

Call Trace Logs and Monitors - Call Trace Filters Create

The following fields are displayed:

Call Trace Filter parameters

Parameter	Description
`Name`	Specifies the name of the Call Trace filter that you are about to create.
`State`	Specifies the state of this filter. The options are: `Enabled:` If enabled, this filter can be used for Call Trace. `Disabled (default):` If disabled, it is not used for Call Trace.
`Capture type`	Specifies the type of logs that need to be captured while running the Call Trace. The options are: `Capture trace information (.trc logs) only.` `Capture trace information (.trc logs) and media information (.pkt logs).` The option "Also save CLI call details and statistics for traced calls (Call Detail Status, Call Media Status, Call Resource Detail Status)" is not supported.
`Detail level to log`	Specifies the level of details that need to be capture in the log file when the trace is complete. The options are: `Level 1 - Everything` `Level 2 - Everything but raw hex dump.` `Level 3 - Only external message information and errors.` `Level 4 - Only SIP PDU events.` Note: The Level 4 log is implemented to capture SIP Ladder diagrams for the interpretation of calls. Level 4 Call Trace Filter conditions/behavior: Level 4 traces all the SIP PDU events (without logging other trace messages) at line-rate without dropping any PDUs. These traces are used by SIP Ladder feature for further analysis. The Level 4 filters are based only on `Peer IP Address` and it must be specified to trace calls. This call trace method does not trace/filter based on trunk group, called number, calling number, and so on. Configure a Level 4 filter with a wildcard `Peer IP Address` of 255.255.255.255 to match ALL packets to/from any IP address. Level 4 filters identify GCID when possible. However, some messages do not contain GCIDs and may have 0xfffffffff as the GCID (registration, notify, options ping, and so on). No other trace filters can be running at the same time as the level 4 trace (that is, do not run a level 1, 2, or 3 trace at the same time as the level 4 trace.)
`Capture calls that match these filters`	Specifies the criteria that need to be captured in the logs in a Call Trace (these are optional). The options are: `Called Number:` Specifies the called party number to trace. An X or x means accept anything in that digit position. For example, 617xxx1212 would filter for all numbers 6170001212 through 6179991212. The % symbol acts as a wildcard for all remaining digits. For example, use 978% to trace all calls with a 978 prefix. Field Length is up to 30 characters. `Calling Number:` Specifies the calling party number to trace. An X or x means accept anything in that digit position. For example, 617xxx1212 would filter for all numbers 6170001212 through 6179991212. The % symbol acts as a wildcard for all remaining digits. For example, use 978% to trace all calls with a 978 prefix. Field Length is up to 30 characters. NOTE: The following characters are accepted by the SBC when specifying the Call Filter search criteria : Digits 0-9 Plus (+) symbol Wilcard characters (“x” or “X”) to match any single digit within the search string Wilcard symbol (%) to match any digits from that point forward `Contractor Number:` Specifies the contractor number to trace. An X or x means accept anything in that digit position. For example, 617xxx1212 would filter for all numbers 6170001212 through 6179991212. The % symbol acts as a wildcard for all remaining digits. For example, use 978% to trace all calls with a 978 prefix. Field Length is up to 30 characters. `Redirecting Number:` Specifies the redirecting number to trace. An X or x means accept anything in that digit position. For example, 617xxx1212 would filter for all numbers 6170001212 through 6179991212. The % symbol acts as a wildcard for all remaining digits. For example, use 978% to trace all calls with a 978 prefix. Field Length is up to 30 characters. `CDDN number:` Specifies the Called Directory Number to trace. Field Length is up to 30 characters. `Transfer capability:` Specifies the transfer capability of the call trace filter. Options are: `Audio31Khz` - 3.1 kHz Audio - ITC 3.1 kHz audio calls are traced. The default setting is disabled. `Speech` - ITC Speech based calls are traced. The default setting is disabled. `Unrestricted` - Calls with ITC Unrestricted are traced. The default setting is disabled. `Unrestricted With Tones` - Unrestricted Digital Information With Tones Announcements - Calls with ITC Unrestricted Digital Information With Tones Announcements are traced. The default setting is disabled. `video` - Calls with ITC Video are traced. The default setting is disabled. `Trunk Group:` The name of a trunk group to trace. This string must match the trunk group name exactly. You cannot use wildcard characters for this argument. Field Length is up to 23 characters. `Peer IP Address:` Specifies the peer IpV4 or IpV6 Address for call tracing.
`stop match`	Specifies the criteria to stop the Call Trace. The options are: `When a match occurs in this filter, stop trying to match the other filters` `Continue to try to match up to two other filters after a match is found in this filter` (default)

To Edit Call Filter

To edit any of the Call Filter entries in the list, click the radio button next to the specific Call Filter entry. The Edit Call Trace Filter window is displayed below.

Call Trace Logs and Monitors - Call Trace Filters Edit

Make the required changes and click Save at the right hand bottom of the panel to save the changes made.

To Copy Call Filter

To copy any of the created Call Filter entries and to make any minor changes, click the radio button next to the specific Call Filter to highlight the row.
Click Copy Call Filter tab on the Call Trace Filters panel. The Copy Call Trace Filter window is displayed along with the field details which can be edited.

Call Trace Logs and Monitors - Call Trace Filters Copy
Make the required changes to the required fields and click Save to save the changes.

To Delete Call Trace

To delete any of the created Call Trace Filters, click the radio button next to the specific Call Trace filter which you want to delete.
Click Delete at the end of the highlighted row. A delete confirmation message appears seeking your decision.
Click Yes to remove the entry from the list.

Call Trace Status and Settings

Call Trace Status and Settings panel allows you to perform a Call Trace. It allows you to Start, Save and Stop the traces on the selected filters.

On the SBC main screen, go to Troubleshooting > Call Trace/Logs/Monitors > Call Trace and Packet Capture > Call Trace. The Configure Trace and Media Packet Capture window appears. The First panel reflects the area to perform a Call Trace.

Call Trace Logs and Monitors - Call Trace Main Window

Before you start a Call Trace, let us examine the conditions and the required fields to execute a Call Trace. The following conditions appear on the Call Trace Status and Settings panel.

Call Trace Status and Settings parameters

Parameter	Description
`Call Trace Duration`	Specifies the duration for the Call Trace to run on the selected filter. The options are: `Run trace until stopped (by clicking the Stop Trace button):` If selected, the trace will keep running till it is stopped manually (by clicking the Stop Trace button)`.` `Stop trace after __ minutes:` If selected, add the number of minutes for the Call Trace to run. The value ranges from 1 to 360 minutes and the default value is 360 minutes.
`Number of Matches (optional)`	Specifies the number of times the Call Filter criteria should match before stopping the trace. The value ranges from 1 to 64 times.
`Trace Calls with errors of the type`	Use this object to specify the system-wide call trace error filter trigger criteria. `none` (default)—No SIP PDU logging for this error filter. `Any`—Log all SIP PDUs that meet the requirement of the Parse Error, Out of Dialog, and Early Attempt options below. `Parse Error`—Log received SIP PDUs that fail parsing. If an error is detected by SIP signaling, the accompanying PDUs will not be associated with any dialog, including an existing dialog. Hence all in dialog and out of dialog PDUs are assigned to this category. If an error is detected at the front end pre-parser, the PDU is logged by this software and not forwarded to SIP signaling. This prevents an offending PDU from being logged by both the pre-parser and SIP signaling. `Out of Dialog`—Log incoming syntactically correct SIP PDUs (except INVITE) that do not belong to an existing dialog. `Early Attempt`—Logs incoming SIP PDUs that result in early termination of the call (by either the pre-parser or SIP signaling). Note The system-wide error filter triggers a call trace based on the selected type of error. If you select to trace calls with errors as well as define call filters to trace, the error filter runs in parallel with the call filters. Both may trigger producing a double-PDU output, one for the conditions matched in the call filter, and one for the detected error. For further details on error logs, see the section Call Trace Log Messages.
`Call Trace Status`	Displays the current status of the Call Trace. When there are no trace running, the application states Call trace stopped with a red dot. when a trace is running, the application states Call trace running with a green dot.

To Run Call Trace

Follow these simple steps to run a Call Trace on a selected filter:

Create a Call Trace Filter with the required parameters and conditions mentioned. Refer to To Create Call Filter section to activate the required parameters.
The newly created filters are displayed in the Call Trace Filter panel.
In the Call Trace Status and Settings panel, choose the required conditions to run the trace. Refer to Call Trace Status and Settings to know more on the conditions.
Click Save & Start Trace on the Call Trace Status and Settings panel to start the trace. The trace status is displayed in the same panel.

Note:

When you start the trace, by default, the first three enabled filters in the table are used for the trace.

Call Trace Logs and Monitors - Call Trace Running
The trace stops automatically if the condition is enabled else you have to stop the trace manually by clicking Stop Trace.

Call Trace Logs and Monitors - Call Trace Stop

Once the trace is complete, the logs are generated and stored at Log Management. Look for TRC and PKT log files (.trc and .pkt log files). For more details on log messages, see the section Call Trace Log Messages.

Call Trace Log Messages

The memory available for Call Trace Log messages are limited to 1800 bytes per log entry. If a log entry occupies memory more than the 1800 bytes allocated for it, the entry is divided into two entries. The next 1800 bytes are allocated for the new log entry with the same time-stamp and sequence number. This ensures that a human or a program can parse the entries, and if necessary, can reassemble the entries with the same time-stamp and sequence number into one single chunk.

A sample log entry with memory footprint of more than 1800 bytes is illustrated below (observe the comments within /* */):

A sample log entry with memory footprint of more than 1800 bytes

242 12162016 083615.736943:1.02.00.38938.Info .SIPSG:CALL:FILTER=Cisco_Test_Call GCID=0x00000DDB: 14443. TraceCallLogger invoked - From pstProperties: PeerIP/port: 10.100.100.10/5000 LocalIP/port:10.139.160.141/5000 ulMsgType-0x40000004

/* This is a comment for illustration and not part of the actual log entry produced by the SBC. */
/* The line just below this comment indicates the start of an entry in the log file. */

1787 12162016 083615.737177:1.01.00.38939.MAJOR .SIPSG:FILTER=Cisco_Test_Call GCID=0x00000DDB: EXTRA INFO: Ingress: received msg for CallId:804ee913ccfe61c2aa581a72a000 gcid:0xDDB from IP/port:10.100.100.10/5000, Local IP/port:10.139.160.141/5000
RAW PDU:
INVITE sip:+8005555555@abc.company.com:5000 SIP/2.0
From: "AU-User-A" <sip:+8005555556@fgh.company.com>;tag=2ea08b0a-13c4-5853a77f-e2852bca-687922
To: <sip:+8005555555@ijk.company.com>
Call-ID: 804ee913ccfe61c2aa581a72a000
CSeq: 1 INVITE
Via: SIP/2.0/TCP 10.100.100.10:5000;branch=z9hG4bK-1fe0f0-5853a77f-e2852bca-53b01fb3
P-AV-Message-Id: 1_1
History-Info: <sip:+8005555555@lmn.company.com>;index=1, "8005555555" <sip:+8005555555@lmn.company.com>;index=1.1
P-Asserted-Identity: "AU-User-A" <sip:+8005555556@fgh.company.com>
Max-Breadth: 60
P-Charging-Vector: icid-value="AAS:409-910e4e801e6cf3c1a58aac0a072"
Alert-Info: <cid:internal@opq.company.com>;avaya-cm-alert-type=internal
Accept-Language: en
Supported: 100rel,histinfo,join,replaces,sdp-anat,timer
User-Agent: Avaya CM/R016x.03.0.124.0 AVAYA-SM-6.3.18.0.631804
Max-Forwards: 66
Av-Global-Session-ID: 804e0e91-3ccf-4601-b7aa-581a72a00000
P-Location: SM;origlocname="ACM4 R3 AU";origsiglocname="ACM4 R3 AU";origmedialocname="ACM4 R3 AU";termlocname="T2A R3 AU";termsiglocname="T2A R3 AU";smaccounting="true"
Session-Expires: 1800;refresher=uac
Min-SE: 1800
Allow: INVITE,ACK,OPTIONS,BYE,CANCEL,SUBSCRIBE,NOTIFY,REFER,INFO,PRACK,PUBLISH,UPDATE
Contact: <sip:+8005555556@10.100.100.10:5000;transport=tcp>
Content-Type: application/sdp
Content-Length: 253
 
v=0
o=- 8420266 0 IN IP4 10.240.42.55
s=-
c=IN IP4 10.240.42.111
b=AS:64
t=0 0
a=avf:avc=n prio=n
a=csup:avf-v0
m=audio 49080 RTP/AVP 18 127
a=rtpmap:18 G729/8000
a=rtpmap

/* This is a comment for illustration and not part of the actual log entry produced by the SBC. */
/* The line just after this comment indicates the start of a new entry in the log file, but with the same sequence number and time-stamp. */
/* It indicates that the new entry is an extension of the previous entry, after the quota of 1800 bytes of memory was utilized by the previous entry. */

148 12162016 083615.737177:1.01.00.38939.MAJOR .SIPSG:FILTER=Cisco_Test_Call :127 telephone-event/8000
a=fmtp:18 annexb=no
a=ptime:20
a=sendrecv

Messages like INVITE, in combination with SIP Headers like P-Asserted-Identity, P-Location, etc. often tends to produce log entries with memory footprint of more than 1800 bytes per entry.

Additional information about a call is indicated by the EXTRA INFO delimiter (see the example above). Currently, it comprises of the following:

Ingress/Egress indication in text
Indication if the message is received or sent
CallId
Global Call ID (gcid)
from IP/Port, to indicate the source IP and port of the message
LOCAL IP/Port, to indicate the recipient IP and port of the message
RAW PDU, which is the raw PDU in ascii text but not encrypted. The RAW PDU section is the largest in a log entry, and often wraps between log entries. Sometimes, it even spans over multiple files.

Offline Call Trace Analysis Using LX Utility

The TRC files can be analyzed offline using the Sonus LX tool, which generates a SIP ladder diagram from the TRC or DBG files. The LX tool is a Windows PC application available free of charge to all customers and partners with a license agreement, and can be downloaded as part of your software download request. For details, refer to LXDOC.

LX Main User Interface

Space shortcuts

Page tree