# Understanding Docker Logs: A Quick Guide for Developers

> Learn how to access and use Docker logs to monitor, troubleshoot, and improve your containerized apps in this simple guide for developers.

Source: https://last9.io/blog/docker-logs/

Docker logs are central to [monitoring](https://last9.io/blog/observability-vs-telemetry-vs-monitoring/) and troubleshooting containers. They help developers track application behavior, debug issues, and ensure smooth performance. (Most of these commands also work with [Podman](https://last9.io/blog/podman-vs-docker/), a popular Docker alternative.)

In this guide, we’ll explore how to access and use Docker logs effectively, making it easier to manage your applications and improve their reliability.

**Common Terms and Concepts**

For newcomers to Docker, here are some key terms to understand:

- **Docker Container:** A running instance of a docker image
- **Log Entries:** Individual log messages from containers
- **Log Data:** The aggregate of all container logs
- **Container_name:** Unique identifier for each container

## **Understanding Docker Logging Basics**

Docker captures logs from your container through two main streams: stdout and stderr. By default, these logs are managed by the json-file logging driver, which stores them as JSON files on your host system.

Here's a simple example of running a container and viewing its logs:

```
# Run a Node.js application in a container
docker run -d --name my-node-app node:latest npm start

# View the logs
docker logs my-node-app

# Follow the logs in real-time
docker logs -f my-node-app
```

### **Where Are Docker Logs Stored?**

By default, Docker stores container logs in:

- **Linux:** `/var/lib/docker/containers/<container_id>/<container_id>-json.log`
- **Windows:** `C:\ProgramData\Docker\containers\<container_id>\<container_id>-json.log`

I recently debugged a disk space issue where these log files were consuming too much space. Here’s how I configured log rotation:

```
{
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "10m",
    "max-file": "3"
  }
}
```

## **Docker Engine and Container Management**

I interact with the Docker engine and Docker daemon almost daily. Understanding how these core components handle logging is crucial. The Docker daemon manages all aspects of containers, including their logs.

When working with running containers, you'll frequently use these commands:

```
# List running containers
docker ps

# Get container details
docker inspect <container_name>

# Check logs of a specific container
docker container logs <container_name>
```

The `docker container logs` command is my go-to tool for viewing log output from any specific container. The CLI provides various options to customize how you view these log messages.

### **Environment Setup**

Your Docker environment configuration lives in the `daemon.json` file, where you can specify logging preferences:

```
{
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "10m",
    "max-file": "3"
  }
}
```

The Docker host can handle multiple logging drivers, and you can set environment variables to control logging behavior:

```
# Set environment variables for logging
export DOCKER_LOG_LEVEL=info
```

### **API and Plugin Integration**

The Docker API provides programmatic access to container logs, which is useful for custom tooling. Many monitoring solutions use this functionality to aggregate logs. You can extend logging capabilities using a plugin system that supports various backends.

### **Working with Unix Systems**

On Unix systems, the Docker service typically stores logs in specific locations. The following command shows how to access these logs:

```
# View service logs
docker service logs <service_name>
```

### **Log Format and Configuration**

Logs are stored in JSON format by default. Each log entry contains:

- Timestamp
- Stream (stdout/stderr)
- Log message

When viewing logs, you can specify the number of lines to display:

```
# Get last N lines
docker logs --tail <N> <container_name>
```

## **Advanced Logging Configuration**

### **Log Drivers**

Docker supports various logging drivers. Here are the most common ones I've used:

- `json-file` (default)
- `syslog`
- `journald`
- `fluentd`
- `elasticsearch`

Example of using the syslog driver:

```
docker run --log-driver=syslog nginx
```

### **Filtering and Formatting Logs**

Here are some powerful filtering techniques I use daily:

```
# Get logs since last hour
docker logs --since 1h my-container

# Get last 100 lines
docker logs --tail 100 my-container

# Filter by timestamp
docker logs --since 2024-03-01T00:00:00Z my-container

# Format output as JSON
docker logs --details my-container | jq
```

## **Logging Best Practices**

Based on my experience managing production containers, here are crucial best practices:

**Log Rotation:** Always configure log rotation to prevent disk space issues:

```
docker run \
  --log-opt max-size=10m \
  --log-opt max-file=3 \
  nginx
```

**Structured Logging:** Use JSON formatting in your applications:

```
// Node.js example
console.log(JSON.stringify({
  level: 'info',
  message: 'User logged in',
  timestamp: new Date().toISOString(),
  userId: 123
}));
```

**Centralized Logging:** For production environments, use the ELK stack:

```
version: '3'
services:
  web:
    image: nginx
    logging:
      driver: fluentd
      options:
        fluentd-address: localhost:24224
        tag: nginx.{{.Name}}
```

## **Troubleshooting Common Issues**

### **Issue 1: Missing Logs**

If you can't see container logs, check:

```
# Verify logging driver
docker inspect --format '{{.HostConfig.LogConfig.Type}}' <container_id>

# Check if logs exist on disk
ls /var/lib/docker/containers/<container_id>/*-json.log
```

### **Issue 2: High Disk Usage**

Monitor log sizes with:

```
du -h /var/lib/docker/containers/*/*-json.log
```

### **Docker Compose Logging**

When using Docker Compose, you can configure logging in your `docker-compose.yml`:

```
services:
  web:
    image: nginx
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
        max-file: "3"
```

## **Monitoring and Alerting**

For production environments, I recommend setting up proper observability:

- Use Prometheus for metrics
- Configure alerting based on log patterns
- Implement log aggregation using Fluentd or Logstash

## **Future-Proofing Your Logging Setup**

As containers move toward Kubernetes environments, consider:

- Using open-source tools that work across platforms
- Implementing [structured logging](https://last9.io/blog/structured-logging/) early
- Planning for scaling log management

## **Conclusion**

Understanding Docker logs is essential for effective debugging and troubleshooting. Start with the basics, implement proper log management early, and scale your solution as needed.

Want to discuss more? Join [our community on Discord](https://discord.com/invite/W8gMppQC4b)! We have a dedicated channel where you can connect with fellow developers and chat about your specific use case.

## **Additional Resources**

- [Docker Logging Documentation](https://docs.docker.com/config/containers/logging/)
- [ELK Stack on GitHub](https://github.com/elastic/elasticsearch)
- [FluentD Documentation](https://docs.fluentd.org/)

## **FAQs**

**Q: How do I view logs for a specific Docker container?**

A: Use the docker logs \<container_name> command. For real-time logs, add the -f flag: docker logs -f \<container_name>.

**Q: Where does Docker store logs by default?**

A: On Linux, logs are stored in /var/lib/docker/containers/\<container_id>/\<container_id>-json.log. The location can be configured in your **config** file.

**Q: How can I change the logging driver?**

A: Modify the **docker logging driver** in your daemon.json file or specify it when running a container using the --log-driver flag.

**Q: How do I handle log rotation?**

A: Configure log rotation options in daemon.json or use the --log-opt flag when running containers.

**Q: Can I get logs from multiple containers at once?**

A: Yes, you can use tools like docker-compose logs or aggregate logs using ELK stack.

**Q: How do I filter logs by date/time?**

A: Use the --since and --until flags with the docker logs command.

**Q: What should I do if logs are missing?**

A: Check the logging driver configuration and ensure the **docker service** is running correctly.

**Q: How can I view logs for a Docker image during build?**

A: Use docker build --progress=plain to see detailed build logs.

**Q: Can I get logs in a specific format?**

A: Yes, use the --format option to customize the log output format.

**Q: How do I handle log storage in production?**

A: Implement log rotation, use external logging solutions, and monitor disk usage regularly.