Introducing Zalando Logbook and How to Integrate It with Spring Boot

Introduction

This guide offers insights into leveraging Zalando Logbook for comprehensive logging in Spring Boot applications. Logbook is an extensible Java library for monitoring HTTP request and response logging. By the end of this post, you’ll understand how to use Zalando Logbook to enhance your logging strategy in Spring Boot environments.

What is Zalando Logbook?

Zalando Logbook is an open-source library designed for transparently logging HTTP requests and responses in Java applications, including Spring Boot. It streamlines the logging process so you can focus on what matters without muddling your business logic. It’s particularly useful for:

• Centralized logging of HTTP traffic.
• Straightforward configuration and setup.
• Customizable to filter sensitive information.

Use Cases for Zalando Logbook

Understanding when and why to use Zalando Logbook can help in envisioning its integration into your projects. Below are some scenarios in which Logbook proves to be particularly beneficial:

• Debugging Complicated Issues: when reproducing bugs is challenging, especially those reported in production environments, detailed logs of HTTP requests and responses can provide insights into the conditions when the error occurs.
• Auditing and Compliance: Keeping track of all transactions is often a requirement for auditing purposes. Zalando Logbook can ensure that every HTTP interaction is logged, providing the necessary audit trail.
• Monitoring API Usage: Using Logbook can help you analyze patterns in API usage. Logging endpoints hit frequency, payload sizes, and user agents enabling you to optimize your APIs performance and security.

Zalando Logbook’s Core Concepts

Zalando Logbook is not only powerful due to its ability to log HTTP transactions; its strength also lies in the flexibility and depth it offers. Let’s delve into some of the core concepts that make Logbook so adaptable to various use cases:

Strategy: Crafting Your Logging Approach

Logbook’s logging strategy is key, determining the extent and timing of logging to fit your app’s needs, with options ranging from minimal to detailed. The logbook comes with some built-in strategies:

• Default Strategy: Logs both the request and response fully.

• Body-Only-If-Status-At-Least-Strategy: This strategy only logs the request/response body if the HTTP status code is greater than or equal to a specific value you configure. This is useful if you want to avoid logging sensitive information in the body for successful requests (typically status codes below 300) but still capture the body for errors (codes above 300).

• Status-At-Least-Strategy: This strategy logs only the request/response headers and the status code, but excludes the message body. This is helpful when you want to keep logs concise or avoid logging potentially large bodies for all messages.

• Without-Body-Strategy: As the name suggests, this strategy completely excludes the request/response body from the logs. You’ll only see headers and status codes. This is useful for scenarios where body content is irrelevant or highly sensitive.

Attribute Extractor: Diving Deeper into Request and Response Data

Attributes are essentially key/value pairs that can be extracted from both requests and responses, these extracted attributes are then logged alongside the corresponding request or response. This is particularly useful for enriching your logs with custom data like user identifiers, session IDs, or transaction IDs. By implementing a custom Attribute Extractor, you can pinpoint exactly what information is crucial for diagnostics, monitoring, and auditing purposes within your application.

Phases: Log Processing Stages

The phases of Logbook provide flexibility and control over what gets logged and how it’s formatted. Here’s a breakdown of the key phases:

1. Conditional Phase: This initial stage determines whether a particular request gets logged entirely. You can configure conditions to exclude specific requests, like health checks that might be pinged frequently.

2. Filtering Phase: After a request passes the conditional phase, filtering allows you to modify parts of the request data before logging. This might involve removing sensitive information or customizing how certain elements are presented.

3. Formatting Phase: Here, the request and response data are shaped into a specific format for logging. Zalando Logbook provides options for formatting the output, but you can also implement your own formatting logic.

4. Writing Phase: Finally, the formatted log message is written to the designated destination. This could be a file, a console, or an external logging system. Zalando Logbook offers various writer implementations, and you can create custom writers as well.

Setting Up Zalando Logbook with Spring Boot

To integrate Zalando Logbook into your Spring Boot application, follow these steps:

Step 1: Add the Zalando Logbook starter dependency to the pom.xml file

<dependency>
    <groupId>org.zalando</groupId>
    <artifactId>logbook-spring-boot-starter</artifactId>
    <version>{latest-version}</version>
</dependency>

Step 2: Configure Logbook in application.yml

For basic logging requirements

logbook:
  format: http
  filter:
    enabled: true
    exclude:
      - /health

For advanced obfuscation and filtering needs:

The configuration can include specifying the log format, which requests to log, and other parameters

logbook:
  format: http                  # Choosing the format as HTTP for readability
  strategy: body-only-if-error # Log request and response bodies only on error status responses
  filter:
    enabled: true
    exclude:
      - /health               # Excluding health check endpoints from logs
      - /metrics              # Excluding metrics endpoints from logs
  obfuscate:
    headers:
      - Authorization          # Obfuscate Authorization header in logs
      - Cookie                # Obfuscate Cookie header in logs
    parameters:
      - password               # Obfuscate password parameters in query strings
    body:
      patterns:
        - '"password": ".*"'   # Use regex to obfuscate 'password' fields in JSON bodies
  write:
    level: TRACE                # Set log level to TRACE for detailed logs
    category: logbook.log      # Define a custom logger category
  max-body-size: 1000          # Limit the maximum body size in logs to 1000 bytes
  exclude:
    content-types:
      - application/octet-stream # Exclude binary data content types from logs
  request:
    conditions:
      methods: [POST, PUT]      # Only log POST and PUT HTTP methods

Zalando Logbook in Action

With the basic configuration, Zalando Logbook will log all incoming and outgoing HTTP requests and responses automatically. Consider this REST controller:

@RestController
public class DemoController {

    @GetMapping("/demo")
    public String getDemo() {
        return "Zalando Logbook is logging this response!";
    }
}

The log output for this would be:

Incoming Request: GET http://localhost:8080/demo
Outgoing Response: 200 OK Zalando Logbook is logging this response!

When set up to obfuscate sensitive information, Logbook’s output for this authentication endpoint:

@RestController
public class AuthController {

    @PostMapping("/api/login")
    public ResponseEntity<?> login(@RequestBody LoginRequest loginRequest) {
        // Simulate an authentication check
        if ("user@example.com".equals(loginRequest.getUsername()) && "secret".equals(loginRequest.getPassword())) {
            // Return a successful response (200 OK) for demonstration, if the credentials match
            return ResponseEntity.ok().body("Login successful!");
        } else {
            // Return a 401 Unauthorized response if credentials do not match
            return ResponseEntity.status(HttpStatus.UNAUTHORIZED).body("Invalid credentials");
        }
    }

    // Simple POJO to represent the login request
    static class LoginRequest {
        private String username;
        private String password;

        // Getters and setters omitted for brevity
        public String getUsername() {
            return username;
        }

        public void setUsername(String username) {
            this.username = username;
        }

        public String getPassword() {
            return password;
        }

        public void setPassword(String password) {
            this.password = password;
        }
    }
}

Would be obfuscated like so:

TRACE 2024-04-23 10:15:30.123 --- [http-nio-8080-exec-1] logbook.log : Incoming Request: POST http://localhost:8080/api/login HTTP/1.1
Authorization: ***** (obfuscated)
Content-Type: application/json
{"username":"user@example.com","password":"*****"} (body content obfuscated)

TRACE 2024-04-23 10:15:30.456 --- [http-nio-8080-exec-1] logbook.log : Outgoing Response: HTTP/1.1 401 Unauthorized
Content-Type: application/json
{"timestamp":"2024-04-19T10:15:30.456+0000","status":401,"error":"Unauthorized","message":"Invalid credentials","path":"/api/login"}

Best Practices and Advanced Tips

Conclusion

Implementing Zalando Logbook in your Spring Boot applications can significantly simplify logging while providing a high level of transparency and control. We hope this introduction and guide encourage you to integrate Zalando Logbook into your projects for better logging and troubleshooting.

Reference

• Zalando logbook document
• Lesson learned – Logbook (youtube)