6 Tips to Improve Your Application Logging

Logs know the deepest and darkest secrets of your application. Learn how to get the best out of them.

Logging is an essential aspect of application development and maintenance. Regardless of the technological stack, logs provide valuable insights into the inner workings of your software, allowing you to diagnose issues, monitor performance, and gain a deeper understanding of your system’s behavior.

In this article, you will find six tips to enhance your application logging practices, so you could get the maximum value out of your logs.

1. Decouple logging from stashing

A key principle of efficient logging is to decouple the logging process from your application. This allows your app to focus on its core functionality while ensuring centralized log ingestion and analysis.

Log ingestion pipeline with Elastic Stack, from source to visualization

The best practice is to use a standard error channel for logging inside the app. After that, you can use dedicated log shipper tools like Logstash, Fluentd, or Filebeat. These log shippers can collect log entries from various sources, apply transformations and then forward them to Elasticsearch or any other log management system. You should avoid sending messages directly to the place where you stash the logs.

In Scoro we use Elastic Stack for logging. We’ve defined the input in our Filebeat configuration as follows:

filebeat.inputs:
- type: container
 paths: 
 — ‘/var/log/containers/*.log’

By utilizing such a configuration, you can easily experiment with various log management systems without having to worry about the application’s logging capability since it’s separated from the app.

2. Use logging libraries

This might be obvious, but avoid reinventing the wheel when it comes to logging. Instead, leverage well-established logging libraries such as Monolog for PHP, Logrus for Go, or Winston for Node. These libraries offer robust features, customization options, and integrations with popular log aggregators. By utilizing such libraries, you can benefit from their extensive community support, bug fixes, and performance optimizations, ensuring efficient and reliable logging for your applications.

3. Use structured logging

Traditional logging often involves concatenating strings to form log messages. However, structured logging takes a different approach by storing log data as key-value pairs or structured objects. This method offers several benefits, including easier log analysis, efficient filtering, and the ability to extract specific fields for analysis.

Let’s imagine we want to debug something. We would write a line like this:

log.Debug(“Foo Bar”);

Traditionally, this would end up looking something like this in a log:

DEBUG 2023-07-12 13:32:28 – Foo Bar

Now let’s compare it to structured logging where the result might look something like this:

[12-Jul-2023 13:32:28 Europe/Berlin] {
  "message": "Foo Bar",
  "context": {},
  "level": 100,
  "level_name": "DEBUG",
  "channel": "error_log",
  "datetime": "2023–07–12T13:32:28.172521+02:00",
  "extra": {},
  "file": {
    "path": "/path/to/log/file",
    "line": 63
  },
  "memory_usage": "18 MB",
  "correlation_id": "f33ff2e1–5b47–8ea4–910d-fee7d68d29b0"
}

This example clearly shows that structured logging gives you a lot more context for your logging event. This extra information might be crucial for debugging or analysis of your application.

Popular logging frameworks like Monolog, Logrus, and Winston provide structured logging capabilities that you can leverage in your applications.

4. Add correlation IDs to log items

In complex distributed systems, it can be challenging to track and correlate log entries across various components. By introducing correlation and tracing IDs to your log items, you can associate related log messages and gain a holistic view of a specific request or transaction. This practice greatly simplifies troubleshooting since it allows you to trace the flow of the execution across multiple services or modules.

Using correlation IDs with microservices

The correlation ID is usually a part of the HTTP request, commonly named as X-Correlation-Id in the header. I will be passed from one microservice to another and get included in the logs. It allows to group together log item generation in one logical flow of requests.

Elastic Stack supports distributed tracing, which makes it easier to visualize and work with these logs.

5. Utilize decorators for exception logging

Exception handling plays a vital role in application resilience and error tracking. Enhance your logging capabilities by employing decorators for different exceptions. By utilizing decorators, you can add additional context, metadata, or stack traces to specific exception types.

An example usage could look something like this:

//declare an interface for decorating 
interface LoggedExceptionDecoratorInterface {
    public function decorateLoggedMessage(array $data): array;
}

//use the interface and include new details to the log
class ORMDriverException extends ORMException implements LoggedExceptionDecoratorInterface {

 public function __construct(private string $query, private array $params, Exception $e = null) {
  parent::__construct($e->getMessage(), $e->getCode(), $e);
 }

 public function decorateLoggedMessage(array $data): array {
  $data['query'] = $this->query();
  $data['params'] = $this->params();

  return $data;
 }

}

//inject deocrated details to the log message
class CustomLogFormatter extends JsonFormatter {

 protected function normalizeException(Throwable $e, int $depth = 0): array {
  $data = parent::normalizeException($e, $depth);

  if ($e instanceof LoggedExceptionDecoratorInterface) {
   $data = $e->decorateLoggedMessage($data);
  }

  return $data;
 }

}

Here we have an interface for the decorator, an example exception using it, and a formatter that will inject additional data into the structured log item. This would result in the following structured log item:

[12-Jul-2023 13:32:28 Europe/Berlin] {
  "message": "Foo Bar",
  "context": {
    "exception": {
      "class": "Scoro\\Exception\\Foo\\BarException",
      "message": "Foo Exception",
      "code": 419,
      "query": "SELECT * FROM products WHERE id = :id",
      "params": {
        "id": 123
      }
    }
  },
  "level": 100,
  "level_name": "DEBUG",
  "channel": "error_log",
  "datetime": "2023–07–12T13:32:28.172521+02:00",
  "extra": {},
  "file": {
    "path": "/path/to/log/file",
    "line": 63
  },
  "memory_usage": "18 MB",
  "correlation_id": "f33ff2e1–5b47–8ea4–910d-fee7d68d29b0"
}

This approach makes it really easy for developers to add important contextual details to logs, without altering the surrounding structures. This enhances the diagnostic capabilities of your logs, allowing faster issue identification and resolution.

6. Settle on a global schema for application logs

We already covered the benefits of structured logging above. In microservices or distributed architectures, it is crucial to maintain a consistent log format. By agreeing upon a global schema for your application logs, you ensure that logs across different services follow a standardized structure.

A really easy option for defining this structure is the JSON Schema. It is a broadly used option and has a wide range of available tooling to help to integrate it into your services. The JSON schema for the example structure log items would look like this:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$ref": "#/definitions/Foo",
  "definitions": {
    "Foo": {
      "type": "object",
      "properties": {
       "message": {
         "type": "string"
       },
       "context": {
         "$ref": "#/definitions/Context"
       },
       "level": {
         "type": "integer"
       },
       "level_name": {
         "type": "string"
       },
       "channel": {
         "type": "string"
       },
       "datetime": {
         "type": "string"
       },
       "file": {
         "$ref": "#/definitions/File"
       },
       "correlation_id": {
         "type": [
          "null",
          "string"
         ]
       },
       "memory_usage": {
         "type": [
          "integer",
          "string"
         ]
       },
       "extra": {
         "$ref": "#/definitions/Extra"
       }
      },
      "required": [
       "channel",
       "correlation_id",
       "datetime",
       "extra",
       "file",
       "level",
       "level_name",
       "memory_usage",
       "message"
      ],
      "title": "Foo Schema"
    },
    "Context": {
      "type": [
       "object",
       "array"
      ],
      "additionalProperties": true,
      "properties": {
       "exception": {
         "$ref": "#/definitions/Exception"
       }
      },
      "title": "Context"
    },
    "Exception": {
      "type": "object",
      "additionalProperties": true,
      "properties": {
       "class": {
         "type": "string"
       },
       "message": {
         "type": "string"
       },
       "code": {
         "type": "integer"
       },
       "file": {
         "type": "string"
       },
       "trace": {
         "type": "array",
         "items": {
          "type": "string"
         }
       }
      },
      "required": [
       "class",
       "code",
       "file",
       "message",
       "trace"
      ],
      "title": "Exception"
    },
    "Extra": {
      "type": "object",
      "additionalProperties": true,
      "title": "Extra"
    },
    "File": {
      "type": "object",
      "properties": {
       "path": {
         "type": "string"
       },
       "line": {
         "type": "integer"
       }
      },
      "required": [
       "path",
       "line"
      ],
      "title": "File"
    }
  }
}

You can also use this schema in your log ingestion pipeline to map out the fields exported and stored in Elasticsearch or implement automated tests as per-service requirements to test if a service provides logs in the proper format.

A standardized log schema simplifies log parsing, analysis, and visualization, especially when using centralized log management tools like Kibana. It provides an unified view of your system’s logs and helps to clearly define how logs should look like across microservices within your organization.

Bonus Tip

Utilize the logging levels described by RFC 5424 effectively. This means that you need to define the severity level for each log message. These are as follows:

[embed]RFC 5424 severity levels

Popular logging frameworks support this out of the box. Once again, this allows you to conduct more granular logging in a standardized fashion.

In conclusion

Effective application logging is a cornerstone of robust software development, providing essential insights into application behavior and aiding in fixing bugs. By employing strategies such as decoupling logging from stashing, leveraging structured logging, and maintaining a global schema, developers can maximize the value of the logs. These practices don’t just foster an efficient debugging environment but also enhance system monitoring and performance analysis capabilities. Therefore, it’s important to follow these suggested practices to improve the overall quality and resilience of your software applications.

If you found this article useful or you just like the benefits of well-instrumented logging and are curious about what else we do in Scoro, then do check out our open positions at https://www.scoro.com/careers/.