What is /var/log: Understanding Linux System Logs

Your Linux system keeps a detailed record of its activity, and most of it lives in one place: /var/log. It’s the go-to directory when something breaks, slows down, or behaves oddly. For anyone working in DevOps—or even just getting started with Linux—knowing your way around system logs isn’t optional. It’s part of the job.

This guide explains /var/log in clear, practical terms: what it is, why it's useful, and how to read these logs to troubleshoot everyday issues more effectively.

What is /var/log?

/var/log is a standard directory in Linux and Unix-like operating systems where system logs are stored. Think of it as your system's journal – it records events, errors, warnings, and other important information about your operating system and applications.

The name itself tells you a bit about its purpose:

• /var stands for "variable" – it contains files that change over time
• /log refers to logging information – the records of system activities

When you're troubleshooting issues, monitoring system health, or just trying to understand what's happening on your machine, /var/log is your first stop.

Why are System Logs Important?

System logs aren't just technical files taking up space on your drive – they're valuable tools that serve several key purposes:

1. Troubleshooting – When something breaks, logs help you figure out what went wrong and why
2. Security monitoring – Logs can reveal unauthorized access attempts or suspicious activities
3. Performance tracking – They provide insights into resource usage and system bottlenecks
4. Audit trails – Logs create records of who did what and when on your system
5. Preventive maintenance – Spotting warning signs before they become major problems

For DevOps beginners especially, learning to navigate and interpret these logs is a skill that will save you countless hours of frustration.

The Structure of /var/log

When you first look inside the /var/log directory, you might feel overwhelmed by the number of files and subdirectories. Let's break down the most common ones you'll encounter:

Key Log Files in /var/log: What Each One Contains and Why It Matters

Different Linux distributions might use slightly different naming conventions, but the general structure remains similar across systems.

How to View and Navigate Log Files in Linux

Let's get practical and look at the commands you'll use to access and read these logs.

Basic Commands for Working with Logs: Quick Reference Guide

To view the contents of a log file, you can use these commands:

# View entire file
cat /var/log/syslog

# View last 10 lines
tail /var/log/syslog

# View last 50 lines
tail -n 50 /var/log/syslog

# Follow log in real-time (great for troubleshooting)
tail -f /var/log/syslog

# Search for specific terms
grep "error" /var/log/syslog

# Combine commands for powerful filtering
tail -f /var/log/syslog | grep "error"

Using Log Viewers for Better Readability: Beyond Basic Terminal Commands

While command-line tools work great, sometimes you want a more user-friendly way to navigate logs, especially as a beginner. Here are some options:

• Use arrow keys to navigate
• Press / followed by a term to search
• Press q to quit
1. Webmin - A web-based interface for system administration that includes log viewing capabilities

lnav - Log file navigator with syntax highlighting and filtering:

# Install lnav
sudo apt install lnav

# Open logs
lnav /var/log/syslog

less - A built-in pager that makes reading large files easier:

less /var/log/syslog

Common Log Formats and How to Read Them

Most log entries follow a similar structure, though details vary between applications. Let's look at a typical syslog entry:

Jan 15 14:23:01 myserver sshd[12345]: Failed password for invalid user admin from 192.168.1.5 port 43210 ssh2

Breaking this down:

• Jan 15 14:23:01 - Timestamp showing when the event occurred
• myserver - The hostname of the machine
• sshd[12345] - The program name and process ID
• Failed password for invalid user admin... - The actual message describing what happened

Understanding this structure helps you quickly scan logs for relevant information.

Troubleshooting Common Issues Using /var/log

Now let's look at how to use logs to solve some typical problems you might encounter.

Scenario 1: Identifying and Addressing Failed Login Attempts

If you suspect someone is trying to access your system, check the authentication logs:

grep "Failed password" /var/log/auth.log

This will show you all failed login attempts, including usernames and IP addresses. If you see many attempts from the same IP, you might want to block it using a firewall.

Scenario 2: Diagnosing Application Crashes Through Log Analysis

When an application keeps crashing, its logs can provide valuable clues:

# For system applications
grep "segfault" /var/log/syslog

# For specific applications
tail -n 100 /var/log/apache2/error.log

Look for error messages, exceptions, or crash reports that might indicate what's causing the problem.

Scenario 3: Resolving System Boot Issues with Boot Logs

If your system isn't booting properly, the boot logs can help identify the problem:

dmesg | less

Look for errors or warnings during the boot process. Hardware issues, missing drivers, or configuration problems often show up here.

Log Rotation and Management

Logs can grow very large over time, which is why Linux uses log rotation to manage them. This process:

• Creates new log files periodically
• Compresses older logs to save space
• Deletes the oldest logs when they're no longer needed

The configuration for log rotation is typically in /etc/logrotate.conf and the /etc/logrotate.d/ directory.

Checking and Customizing Log Rotation Settings for Your Needs

To see how a specific log is rotated:

cat /etc/logrotate.d/apache2

A typical configuration might look like:

/var/log/apache2/*.log {
    weekly
    missingok
    rotate 52
    compress
    delaycompress
    notifempty
    create 640 root adm
}

This means the Apache logs are rotated weekly, kept for 52 weeks, and compressed to save space.

Advanced Log Management with Observability Tools

As your systems grow, manually checking logs becomes impractical. This is where observability tools come in – they help you centralize, analyze, and get alerts from your logs at scale.

Tools for Enterprise-Level Log Management

If you’re looking for a managed observability solution that’s kinder to your budget—without cutting corners on performance—give Last9 a try.

Last9 powers high-cardinality observability at scale for teams at Disney+ Hotstar, CleverTap, Replit, and more. As a telemetry data platform, we’ve monitored 11 of the 20 largest live-streaming events in history.

With native support for OpenTelemetry and Prometheus, Last9 brings together metrics, logs, and traces—optimizing for cost, performance, and real-time correlation across your systems.

Other useful tools include:

• Loki - A lightweight log aggregation system
• Graylog - An open-source log management platform
• Fluentd - A data collector for unified logging

These tools can help you:

• Collect logs from multiple servers in one place
• Set up alerts for specific events or patterns
• Visualize trends and anomalies
• Correlate issues across different systems

Security Considerations for System Logs

Logs contain sensitive information, so it's important to handle them securely.

Essential Security Measures for System Admins

1. Forward logs to a secure log server to preserve evidence even if the original system is compromised
2. Enable log timestamps to maintain an accurate chronology of events
3. Monitor for log tampering as attackers often try to cover their tracks by modifying logs

Set proper permissions:

sudo chmod 640 /var/log/auth.log
sudo chown root:adm /var/log/auth.log

Best Practices for Working with /var/log

To make the most of your system logs, follow these recommendations:

1. Check logs regularly, not just when there's a problem
2. Create a log-checking routine as part of your system maintenance
3. Learn to filter logs to find relevant information quickly
4. Set up automated monitoring for critical systems
5. Document unusual log entries for future reference
6. Back up important logs before they're rotated

Conclusion

Understanding /var/log and how to use it effectively is a fundamental skill for anyone working with Linux systems. As you grow in your Linux journey, your log interpretation skills will improve, making you more effective at managing and troubleshooting systems.

FAQs

How long are logs typically kept in /var/log?

Most Linux distributions configure log rotation to keep logs for weeks or months, depending on their importance. System logs might be kept for 4-8 weeks by default, while security logs might be preserved longer.

Can I delete log files to free up disk space?

While you can delete logs, it's better to configure proper log rotation instead. If you must free up space immediately, target older, rotated logs (files with numbers or dates in their names) rather than the current ones.

Why can't I access some log files even as a regular user?

Many log files contain sensitive system information and are restricted to root or specific system groups. Use sudo to access these protected logs.

What should I do if a log file shows repeated errors?

Don't ignore persistent errors – they often indicate underlying problems. Research the specific error message, check if it coincides with system issues, and consider seeking help if you can't resolve it.

How can I tell if someone has tampered with my log files?

Unexpected gaps in timestamps, missing entries, or logs that are smaller than they should be can indicate tampering. Setting up secure centralized logging helps protect against this.

Can logs affect system performance?

Very active logs with high write rates can impact disk I/O performance. If you notice this issue, consider adjusting logging verbosity or moving logs to a separate disk.