Last9 Last9

Feb 4th, ‘25 / 17 min read

How to Master Zap Logger for Clean, Fast Logs

Learn how to use Zap Logger effectively for clean, fast logs in your applications with this simple, comprehensive guide.

How to Master Zap Logger for Clean, Fast Logs

Today, logging is more than just an afterthought—it’s a crucial part of monitoring, debugging, and maintaining applications. Zap Logger stands out among the many logging libraries available due to its speed, simplicity, and flexibility.

In this comprehensive guide, we’ll explore Zap Logger in-depth, covering its features, benefits, advanced configurations, and best practices to help you get the most out of this powerful tool.

What is Zap Logger?

Zap is a high-performance, structured logging library designed for Go (Golang). It was created by Uber and is known for its lightning-fast log generation, making it a top choice for large-scale distributed systems.

Unlike traditional logging libraries that output simple text logs, Zap Logger produces structured logs, which can be easily parsed and analyzed by other tools, improving efficiency for both developers and systems.

Zap’s primary strengths are:

  • Speed: It's one of the fastest loggers in the Go ecosystem, capable of handling high throughput.
  • Structured Logging: It offers structured log output in JSON format, allowing for easy machine processing.
  • Flexibility: Zap allows custom log levels, output formats, and configurations to suit the unique needs of any application.

Let’s break down the key features and explore how to use Zap Logger effectively.

💡
If you're interested in log parsing, check out our article on The Basics of Log Parsing for more insights.

Key Features of Zap Logger

1. High Performance

Zap is optimized for speed. It outperforms many other logging libraries, such as Logrus and standard loggers, by using a zero-allocation design and performing as much work as possible during compile-time.

This makes it ideal for high-throughput applications where performance is critical, such as real-time systems or microservices architectures.

2. Structured Logging

Unlike traditional unstructured logs, which are just plain text lines, Zap Logger outputs logs in structured formats like JSON. This means each log entry can have additional metadata like timestamps, log levels, request IDs, and custom fields, making it easy to filter and analyze logs later.

Example:

{
  "level": "info",
  "ts": 1612240183,
  "msg": "User login successful",
  "user_id": "12345"
}

This structured output makes Zap perfect for use with log aggregation systems like ELK Stack, Prometheus, or Splunk.

3. Flexible Log Levels

Zap comes with several predefined log levels to indicate the severity of log messages. These are:

  • Debug: Detailed information, typically useful for developers.
  • Info: General operational messages, often used for tracking regular application flow.
  • Warn: Indicates potential issues that are not necessarily errors but may need attention.
  • Error: Used for actual errors that need to be addressed.
  • Fatal: Serious errors that cause the application to shut down.

These levels help categorize logs based on importance and allow you to fine-tune what gets logged at different stages of your application’s lifecycle.

4. Zero-Allocation Logging

A standout feature of Zap Logger is its ability to log messages without allocating memory for each log entry, significantly reducing the strain on garbage collection. This is crucial for high-performance applications where memory management can become a bottleneck.

💡
For a deeper dive into logging in Go, be sure to explore our detailed guide on Logging in Go with slog.

How to Get Started with Zap Logger in Go

Installing Zap Logger

To get started, you'll need to install the Zap package in your Go project. You can do this with the following command:

go get -u go.uber.org/zap

Basic Usage

Once installed, you can begin using Zap in your Go application. Here’s a simple example:

package main

import (
	"go.uber.org/zap"
)

func main() {
	// Create a logger instance
	logger, _ := zap.NewProduction()
	defer logger.Sync() // Flushes any buffered log entries

	// Log an info message
	logger.Info("Application started", zap.String("version", "1.0.0"))
}

In the example above, the logger is configured to use the Production setup, which includes settings suitable for production environments (like JSON formatting). You can also create a Development logger with more human-readable logs during development.

Custom Logger Configurations

Zap allows you to tailor your logger to your needs. You can adjust settings like log format, log level, and output destination. Here's an example of a custom configuration:

package main

import (
	"go.uber.org/zap"
)

func main() {
	// Custom logger setup
	config := zap.Config{
		Level:       zap.NewAtomicLevelAt(zap.DebugLevel),
		Encoding:    "json",
		OutputPaths: []string{"stdout", "logs/app.log"},
	}

	// Create the logger with the custom config
	logger, _ := config.Build()
	defer logger.Sync()

	// Log a message
	logger.Debug("Debugging the application", zap.Int("debug_level", 1))
}

In this case, logs are written to both the console (stdout) and a file (logs/app.log), and the logger is set to log debug-level messages.

Advanced Features of Zap Logger

1. Log Sampling

If your application generates too many logs, you can use Zap’s built-in log sampling feature to limit the volume of logs generated for specific events. This is useful when you have high-frequency logs, like background tasks or network requests, that don’t need to be logged every time.

2. Log Context and Fields

Zap allows you to add context to logs by attaching key-value pairs (fields) to each log entry. This is useful when you need to log additional information about an event, like user details, request IDs, or other metadata.

Example:

logger.Info("Processing request", zap.String("request_id", "abc123"))

This makes it easier to track specific events in your logs, especially when dealing with distributed systems where multiple components interact.

3. Multi-Output Configuration

Zap allows you to configure multiple output destinations. For example, you might want to log in to both the console for real-time monitoring and to a file for long-term storage. You can easily add multiple outputs using Zap's configuration options.

4. Log Rotating Files

For applications that generate a large amount of log data, Zap can be combined with log rotation tools to prevent logs from growing indefinitely. You can use external libraries like lumberjack to manage log file rotation and compression.

import "gopkg.in/natefinch/lumberjack.v2"

// Set up log rotation
logger, _ := zap.NewProduction(zap.AddSync(&lumberjack.Logger{
	Filename:   "/var/log/myapp.log",
	MaxSize:    10, // MB
	MaxBackups: 3,
	MaxAge:     28, // days
}))

5. Zap in Distributed Systems

In complex microservices or cloud-native environments, logs need to be correlated across multiple services. Zap Logger can be integrated with distributed tracing systems like OpenTelemetry to include trace and span IDs in your logs. This allows you to track requests as they flow through various components of your system.

💡
If you're looking to manage log files effectively, check out our guide on Log Rotation in Linux.

How to Customize and Extend Zap Logger for your Application

While Zap Logger is already a powerful tool right out of the box, its true potential comes through when you start to customize and extend it to fit your application’s unique needs.

From creating custom encoders to integrating with external systems, here are some advanced techniques for using and extending Zap to supercharge your logging.

1. Custom Encoders

In Zap, an encoder is responsible for converting log entries into a specific output format, whether it’s JSON, plain text, or any other structure. The default encoder is designed for performance, but you can create custom encoders to suit your requirements.

Why Use Custom Encoders?

You might want to customize how logs are output in specific situations. For example, you might want to:

  • Customize the structure of your logs for compatibility with a particular log aggregation system.
  • Use a different encoding format for different environments (e.g., JSON for production, text for development).
  • Add custom fields or change the way log levels are represented.

How to Create a Custom Encoder

Creating a custom encoder in Zap involves implementing the zapcore.Encoder interface. Here's an example of a simple custom encoder that outputs logs in a different format:

package main

import (
	"go.uber.org/zap"
	"go.uber.org/zap/zapcore"
	"fmt"
)

// CustomEncoder implements the zapcore.Encoder interface
type CustomEncoder struct{}

// EncodeEntry customizes the format of each log entry
func (e *CustomEncoder) EncodeEntry(ent zapcore.Entry, fields []zapcore.Field) (*zapcore.Buffer, error) {
	buf := zapcore.NewBuffer()
	buf.AppendString(fmt.Sprintf("[%s] %s: ", ent.Level.String(), ent.Message))
	for _, field := range fields {
		buf.AppendString(fmt.Sprintf("%s=%v ", field.Key, field.Interface))
	}
	return buf, nil
}

func main() {
	// Use custom encoder in a logger
	config := zap.NewProductionConfig()
	config.Encoding = "json"  // Use JSON encoding for production

	core := zapcore.NewCore(&CustomEncoder{}, zapcore.AddSync(os.Stdout), zapcore.DebugLevel)
	logger := zap.New(core)

	logger.Info("Custom encoding in action", zap.String("key", "value"))
}

This custom encoder outputs logs in a more human-readable format, with each log entry prefixed by its level and message, followed by additional fields.

2. Custom Sinks

In Zap, a sink is an output destination where logs are written. The default sinks are typically stdout (console) or files, but you might want to route logs to custom destinations, such as:

  • A remote server for centralized logging.
  • A third-party service like Last9, Prometheus, or Splunk.
  • An in-memory buffer for performance-sensitive environments.

Why Use Custom Sinks?

Custom sinks allow you to route your logs wherever you need them. Whether you're pushing logs to an external monitoring system or aggregating them for later analysis, Zap’s flexibility makes it easy to add these integrations.

Example of a Custom Sink

To create a custom sink, you need to implement the zapcore.WriteSyncer interface. Here's an example of creating a custom sink that writes logs to an external system (simulated in this case):

package main

import (
	"go.uber.org/zap"
	"go.uber.org/zap/zapcore"
	"log"
	"os"
)

// CustomSink simulates a custom sink for logging
type CustomSink struct{}

func (s *CustomSink) Write(p []byte) (n int, err error) {
	// Simulate writing logs to an external service
	log.Printf("Sending log to external service: %s", string(p))
	return len(p), nil
}

func main() {
	// Create a custom sink
	sink := &CustomSink{}
	core := zapcore.NewCore(zapcore.NewJSONEncoder(zap.NewProductionEncoderConfig()), zapcore.AddSync(sink), zapcore.DebugLevel)

	// Create a logger with the custom sink
	logger := zap.New(core)

	// Log a message
	logger.Info("Logging to custom sink", zap.String("service", "external-service"))
}

In this example, logs are sent to a simulated external service. This concept can be extended to send logs to real systems like a database or cloud logging service.

💡
If you're diving into application logs, check out our detailed guide on Application Logs for more tips.

3. Integration with Distributed Tracing Systems

In modern distributed systems, logs are just one part of the puzzle. Tracing (e.g., using OpenTelemetry or Jaeger) allows you to track requests across multiple services.

Integrating Zap Logger with distributed tracing helps correlate logs with trace data, providing deeper insights into how requests flow through your system.

Why Integrate with Distributed Tracing?

  • Correlation: By linking logs with trace data, you can trace the lifecycle of a request across different services and microservices.
  • Context: Traces provide additional context, like the duration of operations, which helps you diagnose performance issues more effectively.
  • Centralization: Centralized logging systems like Last9 or Prometheus can aggregate both logs and trace data, giving you a unified view of your system’s health.

Example Integration with OpenTelemetry

Here’s how you can integrate Zap Logger with OpenTelemetry to include trace information in your logs:

package main

import (
	"go.uber.org/zap"
	"go.uber.org/zap/zapcore"
	"go.opentelemetry.io/otel"
	"go.opentelemetry.io/otel/sdk/trace"
	"github.com/openzipkin/zipkin-go"
)

func main() {
	// Set up OpenTelemetry tracing
	tp := trace.NewTracerProvider()
	otel.SetTracerProvider(tp)

	// Create a logger
	logger, _ := zap.NewProduction()

	// Start a trace
	tracer := otel.Tracer("example-tracer")
	ctx, span := tracer.Start(context.Background(), "example-span")
	defer span.End()

	// Add trace context to logs
	logger.Info("Logging with trace info",
		zap.String("trace_id", span.SpanContext().TraceID.String()),
		zap.String("span_id", span.SpanContext().SpanID.String()))
}

This code example links each log entry with OpenTelemetry's tracing context, which includes the trace and span IDs. This way, you can trace logs back to their origin and monitor their path across your system.

4. Adding Custom Fields for Context

A powerful feature in Zap is the ability to add custom fields to your logs, which can provide valuable context for understanding application behavior. You can log request IDs, user identifiers, or any other data that might help with debugging or analyzing logs.

For instance, in a web service, you might want to log the user ID with each request:

logger.Info("Handling request", zap.String("request_id", requestID), zap.String("user_id", userID))

This pattern can be used for logging additional metadata that can help with correlating logs from different parts of the system.

5. Custom Log Sampling

If you want to limit how often certain logs are recorded, Zap allows you to configure log sampling. This is useful for high-volume systems where certain events are logged too frequently. For example, you may not want to log every request to an API, but you might want to log 1 in every 100 requests.

Setting Up Log Sampling

You can use zapcore.NewSampler to set up log sampling. Here’s an example:

package main

import (
	"go.uber.org/zap"
	"go.uber.org/zap/zapcore"
)

func main() {
	// Set up sampling for logs
	sampler := zapcore.NewSampler(zapcore.NewJSONEncoder(zap.NewProductionEncoderConfig()), 100, 1)

	// Create a logger with sampling
	core := zapcore.NewCore(sampler, zapcore.AddSync(os.Stdout), zapcore.DebugLevel)
	logger := zap.New(core)

	// Log a message
	logger.Info("This log is sampled", zap.String("context", "sampling"))
}

This example logs only 1 out of every 100 messages, helping to control log verbosity.

💡
To know more about syslogs in Linux, check out our article on Linux Syslog Explained for a deeper understanding.

Simple Tips for Handling Errors with Zap Logger

Effective error handling is one of the key components of building reliable and maintainable applications.

With Zap Logger, you can capture and log errors in a way that’s both informative and actionable, helping developers identify issues quickly and fix them efficiently.

Here’s how you can make the most of Zap’s error-handling capabilities:

1. Logging Errors with Context

When logging errors, it’s important to provide enough context for the person reading the logs to understand what went wrong and where. Zap makes it easy to log errors alongside additional context such as function names, user IDs, request IDs, or other relevant data.

Why It Matters

  • Traceability: When you log errors with context, you make it easier to trace the issue through your application.
  • Actionability: Including enough context helps the developer or operator know exactly what went wrong and where making it easier to fix the problem.
  • Avoid Ambiguity: Without sufficient context, error logs can be vague, leaving the developer guessing about what happened.

Example: Logging Errors with Context

package main

import (
	"go.uber.org/zap"
	"errors"
)

func main() {
	// Create a logger
	logger, _ := zap.NewProduction()

	// Simulate an error
	err := errors.New("database connection failed")
	if err != nil {
		logger.Error("Failed to connect to database",
			zap.String("operation", "db-connect"),
			zap.String("database", "my_db"),
			zap.Error(err),
		)
	}
}

In this example, we log the error alongside specific details about the operation and database, making it clear where the error occurred and what the issue was.

2. Using zap.Error() for Error Logging

Zap provides a zap.Error() field type, which makes it easy to attach error information to your logs in a structured way. This ensures that your errors are captured with the correct type and are consistently logged across your application.

Why It Matters

  • Consistency: Using zap.Error() ensures that errors are logged in the same format and are easy to parse later.
  • Rich Error Information: This method captures not just the error message but the error itself, including any underlying cause, stack trace, and other properties that may help in diagnosing the issue.

Example: Using zap.Error()

package main

import (
	"go.uber.org/zap"
	"errors"
)

func main() {
	// Create a logger
	logger, _ := zap.NewProduction()

	// Simulate an error
	err := errors.New("failed to open file")
	if err != nil {
		logger.Error("File operation error",
			zap.String("operation", "file-open"),
			zap.String("file", "/path/to/file"),
			zap.Error(err),
		)
	}
}

By using zap.Error(), you get all of the necessary details of the error in a structured format, including the error message and any associated fields you choose to log, like operation and file path.

3. Custom Error Handling Logic

In some cases, you may need more fine-grained control over error logging. For example, you might want to classify errors (e.g., distinguishing between network failures and database failures) or take special actions based on error severity.

Why It Matters

  • Error Classification: By categorizing errors, you can prioritize or handle them differently based on their type or severity.
  • Conditional Logging: Custom logic allows you to control which errors get logged based on specific conditions, avoiding unnecessary verbosity.

Example: Classifying Errors

package main

import (
	"go.uber.org/zap"
	"errors"
	"fmt"
)

func logError(logger *zap.Logger, err error) {
	switch {
	case errors.Is(err, ErrNetwork):
		logger.Error("Network error occurred", zap.Error(err))
	case errors.Is(err, ErrDatabase):
		logger.Error("Database error occurred", zap.Error(err))
	default:
		logger.Error("Unknown error occurred", zap.Error(err))
	}
}

var ErrNetwork = errors.New("network failure")
var ErrDatabase = errors.New("database failure")

func main() {
	// Create a logger
	logger, _ := zap.NewProduction()

	// Simulate an error
	err := ErrNetwork
	logError(logger, err)
}

In this example, we have different types of errors, and we log them differently based on their type. This classification ensures that each error is treated appropriately.

4. Handling Panic and Fatal Errors

In situations where your application encounters a critical error, logging it is important, but you might also want to stop the program from continuing. Zap provides the logger.Fatal and logger.Panic methods for these cases.

  • Fatal(): Logs a message and then stops the program by calling os.Exit(1).
  • Panic(): Logs a message and then triggers a panic, which is useful for errors that should terminate the application but still require a detailed log.

Why It Matters

  • Critical Error Handling: For unrecoverable errors, you want to ensure that logs are generated before the program crashes.
  • Graceful Shutdown: By logging fatal or panic errors, you can handle them gracefully before terminating the program.

Example: Using zap.Fatal and zap.Panic

package main

import (
	"go.uber.org/zap"
	"errors"
)

func main() {
	// Create a logger
	logger, _ := zap.NewProduction()

	// Simulate a critical error
	err := errors.New("fatal error: configuration missing")
	if err != nil {
		logger.Fatal("Critical failure", zap.Error(err))
	}
}

In this case, the logger captures the error, logs it with the necessary context, and then immediately terminates the program.

💡
If you're working with system logs, check out our guide on Systemctl Logs for a clearer picture.

5. Error Wrapping and Unwrapping

Sometimes errors are wrapped with additional context to provide more information about the original error. With Zap, you can log wrapped errors and ensure that the full error chain is visible.

Why It Matters

  • Error Transparency: By logging wrapped errors, you ensure that no useful information is hidden in the error chain.
  • Easy Debugging: Including the full stack of errors makes it easier to understand the full scope of the issue.

Example: Logging Wrapped Errors

package main

import (
	"go.uber.org/zap"
	"errors"
	"fmt"
)

func main() {
	// Create a logger
	logger, _ := zap.NewProduction()

	// Simulate an error being wrapped
	originalErr := errors.New("database connection failed")
	wrappedErr := fmt.Errorf("unable to connect to database: %w", originalErr)

	// Log the wrapped error
	logger.Error("Database connection error", zap.Error(wrappedErr))
}

Here, we wrap an error with additional context and log it, preserving the original error and allowing Zap to display both the wrapped error and the original one for full visibility.

💡
If you're working with error logging in Go, check out our guide on Logging Errors in Go with Zerolog for more insights.

Best Practices for Using Zap Logger

1. Structured Logs Are Your Friend

The power of structured logging is that it makes logs easier to query and analyze. Always aim to log in JSON format, as this allows you to extract meaningful insights from your logs more easily.

2. Use Appropriate Log Levels

Use the log levels wisely. Only log detailed Debug information when needed, as excessive logging can affect performance and lead to information overload. Reserve Info for important application events and use Error or Fatal levels for serious issues that require immediate attention.

3. Centralize Your Logs

For larger applications, it’s best to centralize your logs in a log aggregation tool. Zap Logger integrates well with systems like Last9, Prometheus, and Datadog, which allow you to monitor logs from all parts of your infrastructure in one place.

4. Avoid Sensitive Data in Logs

Be mindful not to log sensitive user data (e.g., passwords, credit card details) or other personal information, as logs are often stored in plain text or transmitted across the network. Use zap.String("user_id", user.ID) instead of logging personal data directly.

5. Test and Monitor Logs Regularly

It’s important to test and validate that your logs are providing the information you need. Periodically review the structure and content of your logs, especially as your application evolves.

Conclusion

Zap Logger is a fast, powerful logging tool built to handle even the toughest applications. With this guide, you'll learn everything you need to get the most out of Zap Logger.

💡
And if you’d like to continue the conversation, our community on Discord is always open. We have a dedicated channel where you can discuss your use case with other developers.

FAQs

1. How Do I Configure Zap for Development vs. Production Environments?

Problem: You need different logging configurations for development and production environments to avoid excessive logging in production or missing important logs in development.

Solution: Zap allows you to set up different configurations using zap.NewDevelopment() and zap.NewProduction().

  • zap.NewDevelopment() provides human-readable output, which is ideal for development.
  • zap.NewProduction() provides structured, machine-readable logs, which are optimized for performance in production.

Example:

package main

import (
	"go.uber.org/zap"
	"os"
)

func main() {
	var logger *zap.Logger
	if os.Getenv("ENV") == "production" {
		logger, _ = zap.NewProduction()  // Production logging
	} else {
		logger, _ = zap.NewDevelopment() // Development logging
	}

	logger.Info("This is an info log")
}

This configuration allows you to automatically switch between development and production logging based on your environment settings.

2. Why is My Zap Logger Not Outputting Any Logs?

Problem: If you don’t see any logs even after setting up Zap correctly, it’s often due to logging levels, configurations, or output destinations.

Solution:

  • Check Logging Level: Make sure you’re setting an appropriate logging level (Debug, Info, Warn, Error, Fatal, Panic). By default, Zap logs at the Info level and above.
logger, _ := zap.NewProduction()
logger.Debug("This will not appear in production")  // Will be ignored
logger.Info("This will appear")
  • Output Destinations: If you are logging into a file or remote system, ensure the file path is correct and the necessary permissions are set. If Zap is set to log in to stdout, check if it is being properly displayed.
  • Silent Mode: If you’ve inadvertently set the logger to zap.NewNop(), it won't log anything. Ensure you are not using a "no-op" logger unintentionally.
logger := zap.NewNop() // Will not log anything!

3. How Can I Improve Log Performance in Zap?

Problem: Zap’s performance can degrade if the log output is too complex, especially when logging at lower levels like Debug in high-throughput applications.

Solution:

  • Use zapcore.WriteSyncer Efficiently: Use WriteSyncer to control how logs are written and to optimize performance (e.g., writing logs asynchronously or batching them).
logger, _ := zap.NewProduction(zap.AddCallerSkip(1))
  • Avoid Excessive String Concatenation: Avoid concatenating strings inside log messages. Instead, pass structured fields directly to the logger. This allows Zap to handle the formatting without unnecessary performance overhead.
logger.Info("User action", zap.String("action", "login"), zap.Int("user_id", 123))
  • Log Levels: Adjust your logging level based on the environment. Disable Debug or Trace level logging in production environments to reduce unnecessary computation.

4. How Do I Use Custom Log Output Formats?

Problem: You need to customize the format of your log outputs (for example, JSON, plain text, or something else) based on your project needs.

Solution: Zap supports different formats for logging. By default, it logs in JSON format. However, you can customize the encoder for both development and production configurations.

Example of Custom JSON Output:

package main

import (
	"go.uber.org/zap"
)

func main() {
	// Custom JSON encoder
	cfg := zap.NewProductionConfig()
	cfg.EncoderConfig.MessageKey = "msg"  // Customize the message key

	logger, _ := cfg.Build()

	logger.Info("This is a custom formatted log message")
}

Example of Plain Text Output:

package main

import (
	"go.uber.org/zap"
)

func main() {
	// Development configuration with plain text
	logger, _ := zap.NewDevelopment()

	logger.Info("This is a plain text log")
}

This allows you to control how your logs are output, whether you want them in a structured format (e.g., JSON) or more human-readable (plain text).

5. How Do I Log Panics and Fatal Errors?

Problem: You want to log critical errors and stop execution gracefully.

Solution: Zap provides two methods for handling critical errors: logger.Panic() and logger.Fatal().

  • logger.Fatal() logs the error and then calls os.Exit(1), terminating the program.
  • logger.Panic() logs the error and then triggers a panic, which can be recovered if necessary.

Example of Logging a Fatal Error:

package main

import (
	"go.uber.org/zap"
	"os"
)

func main() {
	logger, _ := zap.NewProduction()

	// Simulate a fatal error
	err := "fatal error: invalid configuration"
	if err != "" {
		logger.Fatal("Application crashed due to configuration error", zap.String("error", err))
	}
}

In this case, the program will terminate immediately after logging the fatal error.

6. How Do I Log Errors with Contextual Information?

Problem: You want to log an error with additional context (e.g., user ID, operation name) to make the log more actionable.

Solution: Use structured logging to capture additional information along with the error.

Example of Logging an Error with Context:

package main

import (
	"go.uber.org/zap"
	"errors"
)

func main() {
	// Create a logger
	logger, _ := zap.NewProduction()

	// Simulate an error with additional context
	err := errors.New("file not found")
	if err != nil {
		logger.Error("Failed to process file",
			zap.String("filename", "data.csv"),
			zap.String("operation", "file-processing"),
			zap.Error(err),
		)
	}
}

This approach ensures that the logs are detailed and contain all necessary context for troubleshooting.

7. Why Are Logs Getting Repeated in Zap?

Problem: You notice that your log entries are being repeated, even when you expect them to be logged once.

Solution:

  • Check for Duplicate Logging Calls: Ensure you’re not logging the same message multiple times in different parts of your code.
  • Disable Repetitive Logging: For error logs that might occur frequently (like network retries), you can use Zap’s WithOptions method to control how the log is written, or you can check for previous error occurrences before logging.

Example of Preventing Duplicate Logs:

package main

import (
	"go.uber.org/zap"
	"errors"
	"sync"
)

var loggedErrors = sync.Map{}

func logOnce(logger *zap.Logger, err error, key string) {
	if _, loaded := loggedErrors.LoadOrStore(key, true); !loaded {
		logger.Error("Logging error once", zap.String("key", key), zap.Error(err))
	}
}

func main() {
	logger, _ := zap.NewProduction()

	// Simulate an error
	err := errors.New("network timeout")
	logOnce(logger, err, "network-timeout")
}

In this example, the sync.Map ensures that the same error is logged only once.

8. How Do I Handle Log Rotation?

Problem: You need to manage log file sizes and handle log rotation automatically to avoid filling up disk space.

Solution: While Zap doesn’t provide built-in log rotation, you can integrate it with other tools like lumberjack to handle log rotation automatically.

Example with lumberjack:

package main

import (
	"go.uber.org/zap"
	"github.com/natefinch/lumberjack"
)

func main() {
	// Set up lumberjack for log rotation
	writer := &lumberjack.Logger{
		Filename:   "/var/log/app.log",
		MaxSize:    10, // Max size in MB before it rotates
		MaxBackups: 3,  // Max number of backup files
		MaxAge:     28, // Max number of days to retain old log files
	}

	// Create the logger with lumberjack
	logger := zap.New(zapcore.NewCore(
		zapcore.NewJSONEncoder(zap.NewProductionEncoderConfig()),
		zapcore.AddSync(writer),
		zap.InfoLevel,
	))

	logger.Info("This log will be written to the rotating file")
}

In this configuration, logs will automatically rotate when they exceed the specified size, keeping your disk usage under control.

Contents


Newsletter

Stay updated on the latest from Last9.

Authors
Prathamesh Sonpatki

Prathamesh Sonpatki

Prathamesh works as an evangelist at Last9, runs SRE stories - where SRE and DevOps folks share their stories, and maintains o11y.wiki - a glossary of all terms related to observability.

X
Topics