Logging for Docker container microservices

About me

I write code

The problem

Tired of typing "grep ERROR" in production? You should, this was done in the 90s.
We are going to talk about logging: "What to log" and "How to log" to have real-time access to logs from the browser.

What is this post about?

This post wants to explain how to set up a centralized logging system for microservices. In order to stop grepping logs in production machines, but view them from the browser.

Tech stack

You should have a container running on docker, FileBeat, ElasticSearch and Kibana.

The process of building Logging for docker container micro-services

The process is simple:
The application running on docker logs to standard output in JSON format.
Docker puts the standard output to the host Syslog.
FileBeat reads the Syslog entry and pushes the logs to ElasticSearch.
You can now see real-time logs on the browser using Kibana

Challenges I faced

The first setup is a little bit long but not difficult. You have to:
configure your app to log in to a JSON format that FileBeat understands.
setup FileBeat, ElastichSearch, and Kibana.
setup the retention for your logs
choose what to log and how to log

Key learnings

You can get rid of browsing application logs on machine files by setting up a centralized logging system with ElasticSearch.

Final thoughts and next steps

Contact me for implementation details and performance tips