In this section:

Overview

The SBC SWe includes a first-level debugging script called sbcDiagnostic that monitors SBC system start-up and checks the SBC status to identify probable causes if the SBC application fails to start. The resultant run log files are dumped to the directory /home/linuxadmin/sbc_diag_logs/. For detailed information, collect any relevant logs from "SYSDUMP".

The sbcDiagnostic script checks the following details:

  • the startup sequence of the SBC (for example, Cloud Int, CPS, LCA, SBC application) and report status of the SBC application.
  • the limited set of errors in the HostCheckCE Nodeapp.latestsys.latest log files and report issue, if the SBC application is not running,
  • if any of the SBC application core dumps is generated after the last boot.  

Example

Failed Scenario (in case of invalid CE/System name):

To check the issues when an SBC application fails to start:

  1. Login as Linuxadmin to an AWS instance.
  2. Execute the following command:

    linuxadmin@vsbc1:~$ sbcDiagnostic

    The following output is displayed:

    ********* Start Cloud SBC Diagnostic ***********
    cloud-init service is active : ok
    cps is initialized : ok
    
    LCA is coming up. Please wait...
    2018-04-27 05:47:07.196 LifeCycleAgent ERROR Invalid CE/System name: vsbcSystemvsbcSystemvsbcSyste
    2018-04-27 05:47:07.196 LifeCycleAgent ERROR Name must match regular expression: ^[A-Za-z]{1}[-A-Za-z0-9.]*[A-Za-z0-9]{1}$ and max length 26
    2018-04-27 05:47:07.196 LifeCycleAgent ERROR Validation failed
    2018-04-27 05:47:07.196 LifeCycleAgent ERROR CE Name or System Name validation failed. Exiting!!!

Copy the Logs

To copy the relevant trimmed sysdump to the /home/linuxadmin/sbc_diag_logs/ directory, execute the following command:

linuxadmin@vsbc1:~$ sbcDiagnostic 1

The following output is displayed:

********* Start Cloud SBC Diagnostic ***********
cloud-init service is active : ok
cps is initialized : ok

LCA is coming up. Please wait...
2018-04-27 05:47:07.196 LifeCycleAgent ERROR Invalid CE/System name: vsbcSystemvsbcSystemvsbcSyste
2018-04-27 05:47:07.196 LifeCycleAgent ERROR Name must match regular expression: ^[A-Za-z]{1}[-A-Za-z0-9.]*[A-Za-z0-9]{1}$ and max length 26
2018-04-27 05:47:07.196 LifeCycleAgent ERROR Validation failed
2018-04-27 05:47:07.196 LifeCycleAgent ERROR CE Name or System Name validation failed. Exiting!!!

********* Start Trimmed SYSDUMP ***********
Neither file /var/log/sonus/sbx/openclovis/CE_Node1.log nor /var/log/sonus/sbx/openclovis/CE_Node2.log are available : failed
Started collecting logs for further investigation.
File /var/log/sonus/lca/lca.log : copied
File /var/log/sonus/sbx/openclovis/app.latest : not available
File /var/log/sonus/sbx/openclovis/sys.latest : not available
File /var/log/sonus/dhcp.log : copied
File /opt/sonus/conf/sbx.conf : copied
File /opt/sonus/conf/bind_nics : copied
File /root/hostcheck.log : not available
File /opt/sonus/conf/userData.json : copied
File /opt/sonus/conf/metaData.json : copied
File /opt/sonus/sbx/lca/sonusMetaVariablesGetDhcp.xml : copied

Please find the list of logs in /home/linuxadmin/sbc_diag_logs/ directory

If SBC application didn't come up. Please look into below list of logs to debug basic issue.
* Check 'lca.log' file that instance are launched with correct userdata/metadata.
* Check 'userData.json/metaData.json' file for more information regarding instance data.
* Check 'hostcheck.log' for hardware requirements related issue.
* Check 'dhcp.log' or 'sonusMetaVariablesGetDhcp.xml' to know that IPs are assigned on all interfaces.
* Check 'bind_nics' file that mgt0, ha0, pkt0 and pkt1 nics are available.
* Check 'route.out' file for route information.
* Check 'sbx.conf' file for SBC configuration.
* Check 'ifconfig.out' file for interfaces information.
* Check 'app.latest|sys.latest|CE_Node1.log|CE_Node2.log' file for sbc application logs.

Please contact Ribbon Communication customer care for support.
One or more logs files are not copied : failed
All Done

Successful Scenario

To check the status of a successful scenario:

  1. Login as Linuxadmin to an AWS instance.
  2. Execute the following command:

    linuxadmin@vsbc1:~$ sbcDiagnostic

    The following output is displayed:

    ********* Start Cloud SBC Diagnostic ***********
    cloud-init service is active : ok
    cps is initialized : ok
    
    LCA is coming up. Please wait...
    
    LCA bringing-up SBC Application. Please wait...
    
    LCA reported SBC application is up. Re-Conforming application status...
    SBC Application is up as standby : ok

Copy the Logs

To copy the relevant trimmed sysdump to the /home/linuxadmin/sbc_diag_logs/ directory, execute the following command:

linuxadmin@vsbc1:~$ sbcDiagnostic 1

The following output is displayed:

linuxadmin@vsbc1:~$ sbcDiagnostic 1
********* Start Cloud SBC Diagnostic ***********
cloud-init service is active : ok
cps is initialized : ok

LCA is coming up. Please wait...

LCA bringing-up SBC Application. Please wait...

LCA reported SBC application is up. Re-Conforming application status...
SBC Application is up as standby : ok

********* Start Trimmed SYSDUMP ***********
Started collecting logs for further investigation.
File /var/log/sonus/lca/lca.log : copied
File /var/log/sonus/sbx/openclovis/app.latest : copied
File /var/log/sonus/sbx/openclovis/sys.latest : copied
File /var/log/sonus/dhcp.log : copied
File /opt/sonus/conf/sbx.conf : copied
File /opt/sonus/conf/bind_nics : copied
File /root/hostcheck.log : copied
File /opt/sonus/conf/userData.json : copied
File /opt/sonus/conf/metaData.json : copied
File /opt/sonus/sbx/lca/sonusMetaVariablesGetDhcp.xml : copied
File /var/log/sonus/sbx/openclovis/CE_Node2.log : copied
File /var/log/sonus/lca/AWSIPSwitchover.log : copied

Please find the list of logs in /home/linuxadmin/sbc_diag_logs/ directory

If SBC application didn't come up. Please look into below list of logs to debug basic issue.
* Check 'lca.log' file that instance are launched with correct userdata/metadata.
* Check 'userData.json/metaData.json' file for more information regarding instance data.
* Check 'hostcheck.log' for hardware requirements related issue.
* Check 'dhcp.log' or 'sonusMetaVariablesGetDhcp.xml' to know that IPs are assigned on all interfaces.
* Check 'bind_nics' file that mgt0, ha0, pkt0 and pkt1 nics are available.
* Check 'route.out' file for route information.
* Check 'sbx.conf' file for SBC configuration.
* Check 'ifconfig.out' file for interfaces information.
* Check 'app.latest|sys.latest|CE_Node1.log|CE_Node2.log' file for sbc application logs.

Please contact Ribbon Communication customer care for support.
All Done

  • No labels