Kubernetes exposes commands to help you debug from the node level without getting into the worker node using SSH. To accomplish this, run the oc debug/kubectl debug command to gain access to host filesystem and networking.

TopicCommand/location
Interface Operations
  • Run tcpdump/tshark on the host interfaces to capture packets
  • Obtain the link status of the host interface.
  • Run ethtool on the parent interface to see the enabled/disabled flags
Container, Kubernetes and Kernel Logs
  • The worker node logs, pertaining to the containers scheduled on the node, are available at:
    /host/var/log/containers/
  • The Kernel logs and Kubernetes logs (like pod creation/deletion, container creation/deletion, probes logs) are available at:
    /host/var/log/journal/

Note: For decoding the journal logs, use the journalctl command.

Host ConfigurationBrowse through the host configuration if you suspect the host configuration.
Debug Pod

The following command should work in most cases to create the debug pod.

oc debug node/mynode -it --image=ubuntu

But in instances where the CPU policy manager is enabled, the command throws an error. In this case, you must use the template to create debug containers/pods on specific nodes.



Sample debug pod yaml: 
apiVersion: v1
kind: Pod
metadata:
  annotations:
    debug.openshift.io/source-container: container-00
    debug.openshift.io/source-resource: /v1, Resource=nodes/worker-11.blr-ocp1.lab.rbbn.com
  creationTimestamp: null
  name: worker-11-debug-container
  namespace: sbc-svt
spec:
  containers:
  - command:
    - /bin/sh
    image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:bbd63f50a9dce72a97d6abb350b0fbaa013c76500fe434ce26a197f230146ae3
    name: container-00
    resources: {}
    securityContext:
      privileged: true
      runAsUser: 0
    resources:
      limits:
        cpu: 100m
        memory: "200Mi"
      requests:
        cpu: 100m
        memory: "200Mi"
    stdin: true
    stdinOnce: true
    tty: true
    volumeMounts:
    - mountPath: /host
      name: host
  hostNetwork: true
  hostPID: true
  nodeName: worker-11.blr-ocp1.lab.rbbn.com
  restartPolicy: Never
  volumes:
  - hostPath:
      path: /
      type: Directory
    name: host

Reference links: