Node Microservices Logging on Kubernetes

Node Microservices Logging on Kubernetes

Motivation

Normally we simply use console.log(…) to write our debug messages. Although this message is piped to the Kubernetes managed container log, in /var/log/containers folder in the Kubernetes node, we might want to have custom formatted message, and transported to some other channels, for example File, Kafka, or even ELK.

In this article, I will share my technical experience how we make use of Winston (https://github.com/winstonjs/winston) as our Node logging framework.

Let’s Get Started

First thing to do, add winston into our dependencies npm i winston. Next, write logger helper module, which we name is as winston.js . You can download the file here (https://github.com/CyderSG/kuberneteslogging/blob/master/winston.js).

The winston.js module assumes of the existence of LOGPATH and LOGLABEL environment variables. LOGPATH is the local path where to write the log files. LOGLABEL determines the name of the module printed in the log itself. This is an example how to set the environments variables in shell script

export LOGPATH=logexport LOGLABEL=EXPENSE

Then you can just simply use this module in the node JS files, by importing the module var logger = require(‘./winston’); and then logger.debug(`This is my first logging!`, {data: 'abc'}))

Configure the Deployment and Pod in Kubernetes

In this example, we use Rancher as our Kubernetes platform, so the screenshots and instructions are based on Rancher.

First, we need to create Persistent Volume and add Volume Claim in our namespace.

Once done, we can map the Volume to the /log mounted path of our

And of course, we need to register two (2) environment variables

Testing

Let’s test now. We hit our API, then we can see the files created in the Node host.

root@cyder-rancher-worker-node3:/mnt/volume_sgp1_01/djarvis/djarvis-log/common-api# ls -latotal 16drwxr-xr-x 2 root root 4096 Dec 3 14:17 .drwxrwxrwx 8 root root 4096 Dec 4 00:43 ..-rw-r--r-- 1 root root 328 Dec 3 14:29 log.json-rw-r--r-- 1 root root 226 Dec 3 14:29 log.txt

This is the content of the files

root@cyder-rancher-worker-node3:/mnt/volume_sgp1_01/djarvis/djarvis-log/common-api# cat log.json{"message":"This is my first logging!, {data: 'abc'}","level":"info","timestamp":"2019-12-03T14:17:52.586Z","label":"COMMON"}{"message":"This is my first logging!, {data: 'abc'}","level":"info","timestamp":"2019-12-03T14:29:20.541Z","label":"COMMON"}

Please give us your feedback / comment.