This article shares how to make use of Avro to publish and consume typed-messages to and from Kakfa broker. We begin with creating Avro schema in this content { "namespace": "sg.com.cyder.kafka.log.avro", "type": "record", "name": "ApiLog&

MotivationNormally we simply use console.log(…) to write our debug messages. Although this message is piped to the Kubernetes managed container log, in /var/log/containers folder in the Kubernetes node, we might want to have custom formatted message, and transported to some other channels, for example File, Kafka, or

MotivationIn my previous article https://medium.com/@fkarnagi/node-microservices-logging-on-kubernetes-c1768701fddb, I shared how to enable and configure Winston-based NodeJS Microservices logging in Kubernetes, powered by Rancher. In this article, I am going to share my experience on how to publish and logs captured by Winston, and send it to Kafka. This

MotivationWe have some applications running on Rancher Kubernetes platform and we have configured the Kubernetes containers logs to be shipped to Kafka Topics. Right now, we are going to developed a custom Kafka SinkConnector to parse the log records from Kafka Topics to ElaksticSearch. First we need to identity which