You are viewing an old version of this page. View the current version.

Compare with Current View Page History

Version 1 Next »

In this section:

Overview

Use the Event Log object to create, configure, disable and enable system and subsystem level log files to capture system, security, debug, packet, trace and accounting events.

Event Types

EventFacility
System16local0
Debug17local1
Trace18local2
Security19local3
Audit20local4
Accounting22local6
Platform Audit Logs23local7
Console log lpr
SFTP log ftp
Kern Log kern
User Log user
Daemon Log daemon
Auth Log auth, authpriv
Syslog Log news
NTP Log uucp
Cron Log cron
FIPS Log local5

 

For each event type, an event class (subsystem) and severity threshold can be configured. Event classes include:

  • Audit
  • Call processing
  • Directory services
  • Network management
  • Policy
  • Resource management
  • Network routing
  • Platform Rsyslog
  • Security
  • Signaling
  • System management
  • Call trace

The ROLLFILE facility provides a means of closing the active log file and opening a new one with an incremented (name) suffix. This facilitates real-time analysis of system events by performing the analysis on closed, rather than opened and growing, files.

For more information on SBC's support for remote syslog servers and the supported log types, refer to Supported Log Types.

The Event Log object allows you to create event log filters to capture debug, security, system, trace, and accounting events using following parameters:

  • Filter Admin – Filter configuration for each event log type and event class
  • Filter Status – View filter status per each event log type and event class (using the request command)
  • INFO Level Logging Enable – Re-enable INFO level logging if it becomes disabled due to system congestion
  • Memory Usage – Measure memory usage of each process
  • Platform Audit Logs – View platform audit logs of administrative, privileged, and security actions
  • Platform Rsyslog – Method of sending event messages to a syslog server.
  • Subsystem Admin – Filter configuration for each subsystem
  • Type Admin – Event log for configuration items related to each event log type

Note

For security protection, the Netconf interface does not support "/aaa" records.

The page ALLDOC:SBC Core Default/Custom Groups and Passwords was not found  -- Please check/update the page name used in the MultiExcerpt-Include macro

Filter Admin


SBC records the maximum number of Debug Event logs, which can potentially cause memory to become congested resulting in unexpected or undesirable SBC performance.

If using INFO filter level is needed for troubleshooting, the SBC triggers the alarm sonusCpEventLogFileDebugLevelInfoNotification any time the Debug Event Log filter level is set to INFO as a reminder of potential memory congestion due to the accumulation of a large number of Debug Event logs in memory. The alarm includes a warning message to set the filter level to MAJOR. The alarm is enabled or disabled using both CLI and EMA

When the filter level is set to INFO, the following events occur:

  • The SBC generates the alarm sonusCpEventLogFileDebugLevelInfoNotification every five minutes.
  • The SBC generates a warning message Debug Event Log filter level is set to INFO. Set to MAJOR if finished troubleshooting on the last modified Debug Event Log file.

Once the troubleshooting is completed, set the filter level to MAJOR. The alarms are cleared when the filter level is set to MAJOR.

When the filter level is changed, the clear alarm sonusCpEventLogFileDebugLevelInfoClearNotification is triggered and a message Debug Event Log filter level is no longer set to INFO is displayed in the log file.

 

Command Syntax

% set oam eventLog filterAdmin <node name>
	<event_type: audit | debug | memusage | security | system | trace>
	<event_class: audit | callproc | directory | netmgmt | policy |  resmgmt | routing | security | signaling | sysmgmt | trace>
	level <info | major | minor | noevents>
	state <off | on>

Command Parameters

Filter Admin Event Log Parameters

Parameter

Description

filterAdmin

Event Log Class Filter configuration table.

<node name>

SBC node name.

<event type>

The type of event log to configure:

  • audit – System audit data. These files contain a record of all management interactions that modify the state of the system, and includes all the changes made via the CLI and the Netconf interface. These files use .AUD extensions.
  • debug – System debugging data. These files have .DBG extensions.
  • memusage – Process heap memory usage data. These files have .MEM extensions.
  • security – Security level events. These files have .SEC extensions.
  • system – System level events. These files have .SYS extensions.
  • trace – System trace data. These files have .TRC extensions.

<event class>

For each event type, configure one of the following event:

  • audit – Audit subsystem.
  • callproc – Call Processing subsystem.
  • directory – Directory Services subsystem.
  • netmgmt – Network Management subsystem.
  • policy – Policy subsystem.
  • resmgmt – Resource Management subsystem.
  • routing – Network Routing subsystem.
  • security – Security subsystem.
  • signaling – Signaling subsystem.
  • sysmgmt – System Management subsystem.
  • trace – Call Trace subsystem.

level

Minimum severity level threshold for event logging:

  • critical – log only critical events.
  • info – log all events.
  • major – log major and critical events.
  • minor – log all events other than info.
  • noevents – do not log any events.

Note:  Info level logs which are traps or faults are always reported in the system logs.

state

Administrative state of event logging for this event type. Set to “on” if filter entry should take precedence over per-node settings.

  • off (default) – Logging is not activated.
  • on – Logging is activated.

 

Filter Status

Command Syntax

% request oam eventLog filterStatus <node name>
   <event_type: audit | debug | memusage | security | system | trace>
   <event_class: audit | callproc | directory | netmgmt | policy |  resmgmt | routing | security | signaling | sysmgmt | trace>
   resetStats

Command Parameters

Filter Status Event Log Parameters

Parameter

Description

filterStatus

Event log class filter status table.

<system name>

SBC system name.

<event type>

The type of event log:

  • audit – System audit data. These files contain a record of all management interactions that modify the state of the system, and includes all the changes made via the CLI and the Netconf interface. These files use .AUD extensions.
  • debug – System debugging data. These files have .DBG extensions.
  • memusage – Process heap memory usage data. These files have .MEM extensions.
  • security – Security level events. These files have .SEC extensions.
  • system – System level events. These files have .SYS extensions.
  • trace – System trace data. These files have .TRC extensions.

<event class>

Event class for each event type:

  • audit – Audit subsystem.
  • callproc – Call Processing subsystem.
  • directory – Directory Services subsystem.
  • netmgmt – Network Management subsystem.
  • policy – Policy subsystem.
  • resmgmt – Resource Management subsystem.
  • routing – Network Routing subsystem.
  • security – Security subsystem.
  • signaling – Signaling subsystem.
  • sysmgmt – System Management subsystem.
  • trace – Call Trace subsystem.

resetStats

Use this control to reset the value of Events Filtered column of the filterStatus display.

 

INFO Level Logging Enable

The active and standby SBC are designed to turn off INFO level logging if the system becomes congested. The "request oam eventLog infoLevelLoggingEnable clearInfoLevelLoggingDisabled" command is used to re-enable INFO level logging once it is disabled. See sonusCpEventLogInfoLevelLoggingDisabledNotfication - MAJOR for associated trap details.

To view INFO LEVEL LOGGING DISABLED state, run the following command.

'show table oam eventLog typeStatus' Example
> show table oam eventLog typeStatus
                                                                                                                    INFO
                                               TOTAL                                                                LEVEL
          CURRENT      FILE     FILE    TOTAL  FILE      FILES    NEXT      LOG                                     LOGGING
TYPE      FILE         RECORDS  BYTES   FILES  BYTES     DROPPED  ROLLOVER  DESTINATION  LAST FILE DROP             DISABLED
------------------------------------------------------------------------------------------------------------------------------
system    1000005.SYS  216      31756   32     1032744   0        0         localDisk    0000-00-00T00:00:00+00:00  false
debug     1000014.DBG  1601     188964  32     27489838  0        0         localDisk    0000-00-00T00:00:00+00:00  false
trace     1000005.TRC  0        128     32     5224      0        0         localDisk    0000-00-00T00:00:00+00:00  false
acct      1000085.ACT  1        202     32     7592      0        0         localDisk    0000-00-00T00:00:00+00:00  false
security  1000005.SEC  7        1047    32     23610     0        0         localDisk    0000-00-00T00:00:00+00:00  false
audit     1000005.AUD  1002     186238  32     4267027   0        0         localDisk    0000-00-00T00:00:00+00:00  false
packet    1000005.PKT  0        128     32     872       0        0         localDisk    0000-00-00T00:00:00+00:00  false 

Command Syntax

% request oam eventLog infoLevelLoggingEnable clearInfoLevelLoggingDisabled

Command Parameter

Info Level Logging Enable Event Log Parameter

Parameter

Description

clearInfoLevelLoggingDisabled

Use this command to re-enable info level logging after it becomes disabled due to system congestion. If this command is executed while the system is still congested, this may cause the system to become further congested.

Note: Only issue this command once system congestion dissipates. The system may become further congested if this command is executed while the system is still congested

Memory Usage

The SBC Core uses the OAM Event Log memusage command to log the memory usage of each process over a configurable interval. The SBC generates a memory log which is uses to capture and log process heap memory usage over time.

The following limitations apply in this release: 

  • Memory consumption through interval statistics are not reported.
  • Memory usage is reported at the process level, not for individual threads/tasks.

The number of bytes used by an active process are captured in the memory usage log file:

Processes are identified by the log entries encoded by the system. For example, the format of a log entry:
113 03282017 073341.007995:1.01.00.00006.MAJOR .PRS: memusage: 1516445696

The memory usage details are logged to the hard drive in the directory: /var/log/sonus/sbx/evlog 

Note

Use the log number to locate the correct log file. For example:

/var/log/sonus/sbx/evlog/<log number>.mem

where the <log number>.mem is the memory usage log file.

Command Syntax

% set oam eventLog process memusage
        state <enable | disable>
        level <summary | detailed>
        interval <0...140>

 

Command Parameters

Memory Usage Parameters

ParameterLength/RangeDescription
memusageN/AThe peer process memory usage configuration details.

state

N/A

Enable this flag to measure the memory usage of each active process.

  • disable (default)
  • enable
levelN/A

Specifies the level of details to be displayed.

  • summary (default)
  • detailed
Interval0-1440 minutes

The time interval, in minutes, to elapse between the recording of each memory usage file to the hard drive. (Default = 5)

Note: An interval of 1440 minutes (24 hours) equates to one log entry per day for a process.

Platform Audit Logs

Command Syntax

% set oam eventLog platformAuditLogs
      state <disabled | enabled>

Command Parameters

Platform Audit Logs Parameters

ParameterLength/RangeDescription
platformAuditLogsN/A

Use this object to configure a remote server IP address, port, and protocol type to push the platform audit logs to a remote server.

state

N/A 

Enable this flag to allow platform audit logging of administrative, privileged, and security actions.

  • disabled (default)
  • enabled

 

Platform Rsyslog

Use Rsyslog to configure a remote server IP address, port, and protocol type to push platform logs of administrative, privileged, and security actions to a remote server.

When platformRsyslog is enabled, the /etc/rsyslog.conf file is configured to send the configured platform logs to the remote syslog server. The remote server's /etc/rsyslog.conf file must match the configuration of the SBC to receive platform logs. The SBC automatically adds an Access Control List (ACL) rule to send the audit logs through the application layer to the remote server.

 

The following logs will not be supported: Monit, Mail, Printer, dpkg and the /var/log/messages file.

Note
The ACL rule is removed automatically from the default ACL rules when platformRsyslog is disabled.
Note

For a High Availability (HA) pair, the /etc/rsyslog.conf file is updated both on the Active and the Standby SBCs to push the audit logs to the remote server.

Command Syntax

To create a new Server configuration table:

set oam eventLog platformRsyslog servers server<no> remoteHost<host_ip> protocolType<protocol> port <port>

 

Command Parameters

Note

Ensure the Platform Rsyslog state is set to "disabled" before configuring/re-configuring the IP address, port, and/or protocol type of the remote server.

Parameters for Configuring New Remote Syslog Servers

ParameterLength/RangeDefaultDescriptionM/O

no

1-31Number of server.M
host_ipN/AN/AHost IP of server.M
protocolN/ATCP

The protocol used to send messages to the Remote Server.

  • tcp
  • relp
  • udp 
M
portN/A514Specifies the port used to send messages to the remote Server.M
 

 

Command Syntax

To enable/disable the Rsyslog service for all the Linux Logs:

set oam eventLog platformRsyslog syslogState <disabled | enabled>

 

Command Parameters

Parameters for Configuring New Remote Syslog Servers

ParameterDescription

syslogState

Use this flag to enable/disable the Rsyslog service:

  • disabled (default)
  • enabled
 

 

Subsystem Admin

Command Syntax

Mandatory parameters required to configure an Event log subsystem event type:

% set oam eventLog subsystemAdmin <system_name> <subsys_ID>

Non-mandatory parameters to configure an Event log subsystem event type:

% set oam eventLog subsystemAdmin <system_name> <subsys_ID>
	infoLogState <disabled | enabled>
	maxEventID <0-4.294967295E9>
	minEventID <0-4.294967295E9>

Command Parameters

Subsystem Admin Event Log Parameters

Parameter

Description

subsystemAdmin

Subsystem event logging configuration.

<system_name>

Name of system.

<subsys_ID>

The subsystem/task ID. See Subsystem IDs table below for a list of subsystem IDs. 

infoLogState

Use this flag to enable/disable event logging of INFO level messages to DBG and SYS logs for   the specified subsystem. By default, infoLogSate is enabled for all subsystems.

  • disabled
  • enabled (default)

Note:

  • If infoLogState is disabled for CHM, nothing is written to AUD logs.
  • If infoLogState is disabled for CPX, request commands are not recorded to AUD logs.

Subsystem IDs

aka

arm

asg

brm

cam

cc

chm

cpx

dbl

dcm

debug

dfe

dht

diamc

dnsc

drm

ds

dsa

dtls/srtp

ema

enm

enm_am

enm_test

fm

gcl mbs

gclcomm

gwcm

gwfe

gwsg

h248fe

h323fe

h323sg

ice

iceapp1

iceapp2

iceapp3

iceapp4

iceapp5

iceapp6

iceapp7

iceapp8

icms_test1

icms_test2

ike

im

ipacl

ipm

kfqdn

les

license_sm

lvm

lwresd

mgsg

mim

mrm

mtrm

nim

nrm

nrma

nrs

pathchk

perfs

perfs

pes

pipe

prsnp

rgm

rtm

rtma

sbcintf

scpa

sec

sg

sipcm

sipfe

sipsg

sm

sma

ssa

ssreq

surrreg

trcrt

trm

xrm

 

 

 

Type Admin

Note

The syslog ACL rules are added and removed by enabling/disabling syslogState and configuring the syslog log fields.

Note

To guard against overlogging, the SBC logs up to 4,294,976,295 messages per second in the event logs (configurable with set oam eventLog typeAdmin system diskThrottleLimit), but additional event messages above that threshold are discarded. If log events must be discarded, the SBC writes an error message about the skipped messages in the system (.SYS) log.

Command Syntax

The following syntax applies to the set oam eventLog typeAdmin command:

% set oam eventLog typeAdmin <acct | audit | debug | memusage | packet |  security | system | trace>
   compressionSupport <both | none | only>
   compressionDaysToKeep <1 .. 7>
   compressionCleanupDirectorydiskThrottleLimit <0-4294976295>
   eventLogValidation
   fileCount <1-2048>
   fileSize <256-65535>
   fileWriteMode <default | optimize>
   filterLevel <info>
   messageQueueSize <2-100>
   renameOpenFiles <disabled | enabled>
   rolloverAction <start | stop>
   rolloverInterval <0-31536000>
   rolloverStartTime <time>
   rolloverType <repetitive | nonrepetitive>
   saveTo <none | disk>
   servers <syslogRemoteHost | syslogRemotePort | syslogRemoteProtocol>
   syslogState <disabled | enabled>

Only the Administrator can execute the above command using the audit and security attributes:

% set oam eventLog typeAdmin audit...
% set oam eventLog typeAdmin security...

 

The following syntax applies to the request oam eventLog typeAdmin command:

% request oam eventLog typeAdmin <acct | audit | debug | memusage | packet |  security | system | trace> rolloverLogNow

% request oam filterStatus <card name> <audit | debug | memusage | security | system | trace> 
	<audit | callproc | directory | netmgmt | policy | resmgmt | routing | security | signaling | sysmgmt | trace

Only the Administrator can execute the following commands using the "audit" and "security" attributes:

% request oam eventLog typeAdmin audit rolloverLogNow
% request oam eventLog typeAdmin security rolloverLogNow
% request oam eventLog filterStatus <card name> security security resetStats

Note

The System log displays Info level logs which are traps or faults when the System log filterLevel is configured to log Major and/or Critical events.

Command Parameters

Type Admin Event Log Parameters (set command)

Parameter

Length/Range

Description

typeAdmin

N/A

Event Log configuration table for configuration items related to each Event Log type.

<event_type>

N/A

Specifies the type of event log being configured:

  • acct – System account data. These files have .ACT extensions.
  • audit – System audit data. These files contain a record of all management interactions that modify the state of the system. These files have .AUD extensions. It  includes   all the changes made via the CLI and the Netconf interface. (This attribute is only available to an Administrator)
  • debug – System debugging data. These files have .DBG extensions.
  • memusage – Process heap memory usage data. These files have .MEM extensions.
  • packet – Packet information details. These files have .PKT extensions. If enabled, stores the packet details to .PKT files.
  • security – Security level events. These files have .SEC extensions. (This attribute is only available to an Administrator)
  • system – System   level events. These files have .SYS extensions.
  • trace – System trace data. These files have .TRC extensions.

Note: Syslog is not supported for acct and packet event types.

compressionSupport N/A

Type of compression.

  • both – The SBC generates both compressed and uncompressed CDR files
  • none – For backward compatibility, uncompressed CDR files
  • only – The SBC generates compressed CDR files
compressionDaysToKeep 1-7The number of days to keep compressed files before deleting. Default = 5
compressionCleanupDirectoryN/AAlternate directory name (containing no slashes) under the log file directory from which compressed files are removed after compressionDaysToKeep days
diskThrottleLimit

0-4294976295

Specifies the limit on INFO level messages logged to the disk in one second. A value of 0 disables the limit. The default value is 10000.

Note: For the trace log, if tracing is being performed to capture all of the SIP PDU for all of the calls on the system for use in conjunction with Protect, then this value needs to be tuned to accommodate the maximum call load anticipated for the SBC instance. For example, for a call rate of 1350 cps and assuming 14 messages in a basic SIP call (ingress and egress legs), it would require a total of 18,900 messages. Adding this to the default 10000, the recommendation in this case would be to set the limit at 30,000.

eventLogValidation N/A

Specifies whether the logs at rest for this log type should be cryptographically hashed.

Hashing is only recommended for the security and audit logs. These are the main logs required to triage security issues and do not roll very frequently. Hashing must be disabled for logs that are rolling over frequently as would occur for the trace log if the call rate is 1350 cps and it is being used to capture all SIP PDU's for use with Protect.

If logs are being exported using Rsyslog then there is no need to enable Event Log Validation as the logs are copied off the SBC before they could be modified. Refer to OAM - Event Log - Platform Rsyslog.

  • disabled (default)
  • enabled

IMPORTANT: You must disable this control for any logs which are rolling at a very high rate (e.g. capturing trace logs of all SIP PDUs for use with Protect).

Hash Notes:

  • Hashes are stored in /.../evlog/eventLogValidation/
  • The hash file name format is <evLogfilename>.hash.<keyName>
  • Hashes must be retrieved using SFTP 

fileCount

1-2048

Specifies the number of event log files that will be maintained for this event type. (default = 32).

fileSize

256-65535

Maximum size (in KB) that a single event log file will ever grow to. (default = 2048).

Note: Set the file size to 65535 for trace and account logs when attempting to trace all calls on the system for use with Protect.

fileWriteMode

N/A

Event log NFS write mode. 

  • default – Log data is written as a 1344-byte packet.
  • optimize – Log data is written as a 8000-byte packet. Optimize write mode results in IP fragmentation but yields better throughput.

filterLevel

N/A

Logs every possible event.

messageQueueSize

2-100

The number of event log message entries to buffer before writing to disk. (default = 10).  If capturing all of the SIP PDU messages in the trace log for use with Protect, set this value to 100 for the trace log.

renameOpenFiles

N/A

Enable this flag to append an ".OPEN" extension to accounting and files which are open for writing.

  • disabled (default)
  • enabled

Note:You must enable the global callTrace signalingPacketCapture parameter (set state to "enable") to capture SIP and H.323 packets (Refer to Call Trace - CLI for configuration details).

Once signalingPacketCapture is enabled, any subsequent changes to SBC device configurations or filter information will not be available to signaling packet captures until signalingPacketCapture is reset (state is disabled, and then re-enabled).

rolloverAction

N/A

Event log rollover actions.

  • start – Start rollover action
  • stop – Stop rollover action

rolloverInterval

0-31536000

Event log rollover interval, in seconds.

rolloverStartTime

N/A

Specifies the start time for event log rollover. The format is CCYY-MM-DDTHH:MM:SS. For example: 2010-01-01T01:01:01

rolloverType

N/A

Event log rollover type. 

  • nonrepetitive (default) – The rollover will occur once at the specified single instance.
  • repetitive – The rollover will occur repeatedly at the specified intervals.

saveTo

N/A

Use flag to specify that the events are saved to disk or not saved.

  • disk (default)
  • none

state

N/A

Specifies the requested state of the given Event Log type.

  • disabled – Logging is not activated.
  • enabled – (default) Logging is activated.
  • rollfile

 Accounting logs cannot be disabled.

serversN/A

Configure a remote Rsyslog Server for a single log type:

  • syslogRemoteHost – (0-255) The remote host where the messages are written to the syslog.
  • syslogRemotePort – (1-65,535) Specifies the port to use to send messages to the remote syslog. Default value is 514.
  • syslogRemoteProtocol – The protocol to use to send messages to the remote syslog.
    • relp
    • tcp (default)
    • udp 
syslogStateN/AEnable flag to log events of specified type to syslog. 
  • disabled (default)
  • enabled
Notes on Compression-Related File Naming

For Hardware and SWe-Based Systems

  • The compressed files are named using the following convention:

<System Name>_<timestamp>_xxxxxxx.ACT.gz

...where System Name is the name of the Redundancy group. Example: SBX30_1571352583_1000018.ACT.gz

  • The number of files created and maintained concurrently is unlimited, and is not constrained by the fileCount configured for the accounting log.

 

For N:1 Cloud-Based Systems

  • The compressed files are named using the following convention:

< Hostname i.e. VM Name  >_<timestamp>_xxxxxxx.ACT.gz

You cannot use the system name because, in an N:1 system, multiple instances running in active mode would have the same system name.

The SBC uses the actualCeName as the Hostname because this is the name specified in the user metadata. Example: vsbc1Site1_1571352902_1000003.ACT.gz

  • The number of files created and maintained concurrently is unlimited, and is not constrained by the fileCount configured for the accounting log.

 

For 1:1 Cloud-Based Systems

  • The compressed files are named using the following convention:

<System Name>_<timestamp>_xxxxxxx.ACT.gz

...where System Name is the actualSystemName, as this is the name specified in the user metadata. Example:  vsbcSystem22_1571348519_1000001.ACT.gz

  • The number of files created and maintained concurrently is unlimited, and is not constrained by the fileCount configured for the accounting log.

 

Type Admin Event Log Parameters (request command)

Parameter

Description

typeAdmin

Event Log configuration table for configuration items related to each Event Log type.

<event_type>

Specifies the type of event log to roll over:

  • acct – System account data. These files have .ACT extensions.
  • audit – System audit data. These files contain a record of all management interactions that modify the state of the system. These files have .AUD extensions. It  includes   all the changes made via the CLI and the Netconf interface. (This attribute is only available to an Administrator)
  • debug – System debugging data. These files have .DBG extensions.
  • memusage – Process heap memory usage data. These files have .MEM extensions.
  • packet – Packet information details. These files have .PKT extensions. If enabled, stores the packet details to .PKT files.
  • security – Security level events. These files have .SEC extensions. (This attribute is only available to an Administrator)
  • system – System   level events. These files have .SYS extensions.
  • trace – System trace data. These files have .TRC extensions.

rolloverLogNow

This control is used with request command to perform a roll-over of the specified log immediately.

 

 

  • No labels