Logging ??

 Today we will learn about the following things:

• What is logging ??
• Why do we need logging ??
• How can logging help us ??
• Why suddenly from the last 5-10 years logging has been evolving ??
• Logging tools ??

First thing first, let's start with logging. Logging is human-readable information that is primarily text in nature. Logging is triggered or generated when some kind of event occurs, such event is known as a log event.  Each logging message contains the timestamp, which signifies the time at which the event occurred.

Now let, us understand why we need logging. 

Importance of logging ??

In today's world software all around the world captures the whole market. By definition, Software is a program that is made up of millions of lines of code.  The programmer while writing the code adds the logs at all places to know the code behaviors as well as unexpected ones.  The logs can be of types such as ERROR, WARN, INFO, FATAL, DEBUG, TRACE, etc.  

 NOTE: Logs are also added to know what is going inside the code and for the same INFO type log is added. Similarly, WARN, FATAL, etc types of logs are also added to know the warning and critical nature of the code.

The programmers add the logs so that if something goes wrong in any part of the code, it throws a log with the exact line number which helps the developer to know which line of the code exactly has the problem. Suppose there is no log, then in that case the developer has to go through the whole code to find exactly which part of the code caused the problem or issue. So, adding logs saves huge time of developer and helps in debugging fast. 

In a server, tons of software runs, and if something unexpected happens in the software then it throws ERROR types logs. So, logs help in debugging, providing the context of what's going on inside code, the unexpected condition occurs, identifying the root cause of the issue, determining the cause of performance issues, and many more. 

If we look from another point of view, the log is like an observability tool that tells what is happening inside the software. That's why it is also known as one of the pillars of Observablity. Majorly there are 3 observability tools:

• Logging
• Tracing
• Metrics

So we understand the basics of logging and its importance. 

Why suddenly from last the last 5-10 years logging evolved??

It's because of microservice. Microservice is implemented via containerization instead of virtual machines. The monolithic or single application is divided into multiple small applications known as microservices. Due to which logs come from multiple applications. These small applications talk to each other to make the application complete. If some issue occurs in the application, so to know the exact issue. We need to centralize logs from all small applications to a single place to identify the exact issue. That's where the concept of Log unification came into. 

Log Unification ??

Unification means the act or process of bringing together or combining things or people. The Fluentd principle is based on collecting log events from diverse sources and bringing them together to a centralized tool so that the log events can be processed and sent to endpoints for different use cases.
The benefits of the unification of logs:

• A single technology brings log events together, as a result, log event management becomes easier and more accessible.
• The strength of unification tools is sourcing and delivering log events. 

Unifying logs vs log analytics

• Tools like Splunk and ElasticSearch in the logging space come into the category of log analytics where the main focus is to apply data-analysis techniques such as pattern searching using complex rules. Such processions are often associated with big data and search engine technologies. 
• The strength of log analytics is applying search and computational science to collect logs at a centralized level, not the gathering and routing log events. Log analytics is also known as Log aggregation.
• Whereas, the log unification strength is sourcing and delivering the log events. Log unification is also known as Log Routing. 
• Both technologies have strong event-filtering capabilities but are applied or implemented in different ways.
• Log Routing is when log events are taken and then directed through a middleware tool, such as Fluentd, to the application that needs these log events.

Log aggregation means log events are taken and sent to a central location to be processed. Such as Splunk.

Fluentd

The primary purpose of Fluentd is to capture or collect log events from a diverse range of possible sources or targets. Such as:

•  infrastructure as a network device
• OS
• Custom applications
• Prebuilt applications
• Paas
• Saas

It then sends those events to appropriate tools where the log events can be processed to extract meaning and insights, and possibly trigger events. It doesn’t perform detailed log analytics itself. It also unifies the log events from all the sources of logs, by which we have the opportunity to see the big picture.

As you know I mostly focus on content related to the cloud-native environments. So, we will look out to capture logs from Kubernetes environment. And for the same light version of Fluentd was introduced and that is known as FluentBit. So, next we will learn about FluentBit.