Ship Kubernetes Microservices Logs to Kafka Topics
Motivation
In my previous article https://medium.com/@fkarnagi/node-microservices-logging-on-kubernetes-c1768701fddb, I shared how to enable and configure Winston-based NodeJS Microservices logging in Kubernetes, powered by Rancher.
In this article, I am going to share my experience on how to publish and logs captured by Winston, and send it to Kafka.
This article assumes that you are using Rancher as the Kubernetes platform, as our instructions and screenshots are based on Rancher
Let’s Get to Work
First, after login to you cluster, and open your project, you need to enable the Logging.
Choose Kafka as the logging framework
You need to specify the Endpoint (could be more than one) and specify the topic.
Once enabled, you will be able to see the logs in Kafka Topic. I use Kafka Tool to browse the Kakfa topics from my Mac. You can download the tool here http://www.kafkatool.com/.