Ubuntu var log messages: A Complete Guide for System Admins

Ubuntu keeps a detailed record of what’s happening under the hood, and most of it lives in log files like /var/log/messages. These logs can tell you a lot: system errors, service failures, security events, and more.

This guide walks through where to find these logs, what each one covers, and how to read them when you’re trying to figure out what just went wrong.

What /var/log Messages Tell You About Your Server's Health

Ubuntu's /var/log the directory is home to a collection of log files that record system events, errors, warnings, and general information about what's happening on your machine. Think of it as your system's journal—it writes down everything it does, good or bad.

While traditional Linux distributions typically use /var/log/messages as their main system log file, Ubuntu takes a slightly different approach. Ubuntu primarily uses /var/log/syslog for general system messages and has several other specialized log files for different system components.

A Complete Map of Ubuntu's Log Directory

Ubuntu logically organizes logs, with each file serving a specific purpose. Here's a rundown of the most important log files you'll find in the /var/log directory:

Ubuntu doesn't use /var/log/messages by default, but the information you'd expect to find there is typically in /var/log/syslog.

Terminal Commands and GUI Tools for Efficient Log Analysis

Let's get practical. Here are several ways to access and read your log files:

Using the Terminal

The most straightforward approach is to use terminal commands:

# View the entire syslog file
sudo cat /var/log/syslog

# View the last 50 lines
sudo tail -n 50 /var/log/syslog

# Follow the log in real-time (press Ctrl+C to exit)
sudo tail -f /var/log/syslog

Using Log Viewers

If you prefer a graphical interface, Ubuntu offers several log viewers:

• GNOME Logs: A simple, user-friendly log viewer included with Ubuntu Desktop
• KSystemLog: A more advanced log viewer for KDE environments
• Webmin: A web-based interface for system administration, including log viewing

Understanding Log Message Format

A typical log message follows this pattern:

May 10 13:45:01 server-name systemd[1]: Starting Daily apt download activities...

Let's break this down:

• Date and time: May 10 13:45:01
• Hostname: server-name
• Service/program: systemd[1]
• Message: Starting Daily apt download activities...

Red Flags in Your Ubuntu Logs

Your logs are full of valuable clues about what's happening in your system. Here are some common issues you might spot:

Failed Login Attempts

Check /var/log/auth.log for entries like:

May 10 04:23:15 ubuntu-server sshd[12345]: Failed password for invalid user admin from 192.168.1.100 port 54321 ssh2

This could indicate someone trying to break into your system.

Disk Space Warnings

Look for entries in /var/log/syslog like:

May 10 14:32:01 ubuntu-server kernel: [56789.012345] EXT4-fs warning (device sda1): fs_warning: low on free space, only 1024MB available

Time to clean up or expand your storage!

Service Failures

Watch for messages like:

May 10 15:01:23 ubuntu-server systemd[1]: Failed to start Apache Web Server.

This tells you Apache failed to start—time to check its configuration.

Filtering, Rotating, and Centralizing Ubuntu Logs

When you're dealing with mountains of log data, you need tools and techniques to make sense of it all.

Using grep for Log Filtering

The grep command is your best friend for finding specific information:

# Find all SSH connection attempts
sudo grep "sshd" /var/log/auth.log

# Find all error messages
sudo grep "error" /var/log/syslog

# Find messages related to a specific time
sudo grep "May 10 13:" /var/log/syslog

Setting Up Log Rotation

Logs can grow huge if left unchecked. Ubuntu uses logrotate to manage log file sizes:

# Check logrotate configuration
cat /etc/logrotate.conf

# Check specific service rotation settings
cat /etc/logrotate.d/apache2

A typical logrotate configuration rotates logs weekly, compresses old logs, and keeps them for a set period before deletion.

Centralizing Logs with rsyslog

For multi-server environments, centralizing logs makes monitoring much easier:

# On the central log server, edit rsyslog.conf
sudo nano /etc/rsyslog.conf

# Uncomment these lines
$ModLoad imudp
$UDPServerRun 514

# On client machines, add this line to forward logs
*.* @central-server-ip:514

After making changes, restart rsyslog:

sudo systemctl restart rsyslog

Best Monitoring Solutions for Ubuntu Servers

While basic command-line tools work for occasional checks, proper monitoring requires dedicated tools.

Local Monitoring Solutions

These tools can be installed directly on your Ubuntu system:

• Logwatch: Generates daily reports summarizing log activities
• GoAccess: Real-time web log analyzer with terminal and web interface

Cloud-Native Monitoring

For larger environments, consider these options:

• Last9: A telemetry data platform that excels with high-cardinality observability at scale. It integrates with OpenTelemetry and Prometheus to unify metrics, logs, and traces, giving you real-time insights with predictable pricing based on events ingested.
• Grafana Loki: A horizontally scalable log aggregation system
• ELK Stack: Elasticsearch, Logstash, and Kibana for powerful log analysis
• Graylog: Centralized log management for structured and unstructured data

Strategic Log Management for Ubuntu

After years of working with Ubuntu systems, here are some hard-earned tips:

Security Considerations

• Log access permissions: Restrict log access to authorized users

sudo chmod 640 /var/log/auth.log
sudo chown root:adm /var/log/auth.log

• Log file integrity: Consider using AIDE (Advanced Intrusion Detection Environment) to detect unauthorized log changes

Performance Optimization

• Balance retention vs. storage: Keep enough logs for troubleshooting, but not so many that you fill your disk
• Use appropriate log levels: In production, use warning or error levels to reduce noise

Compliance Requirements

If you're in a regulated industry, your logs may need to meet specific requirements:

• Retention periods: Some regulations require keeping logs for 1-7 years
• Tamper-proof storage: Consider writing logs to WORM (Write Once Read Many) storage
• Encryption: Encrypt sensitive logs, especially when they contain personally identifiable information

Fixing Boot, Hardware, and App Issues with Ubuntu Logs

Let's look at some real-world troubleshooting scenarios:

Case 1: Diagnosing Boot Issues

When your system won't boot properly:

1. Boot into recovery mode
2. Check /var/log/boot.log and /var/log/dmesg
3. Look for lines with "error" or "fail"

Case 2: Identifying Hardware Problems

For hardware issues:

1. Check /var/log/kern.log for hardware-related messages
2. Use dmesg | grep error to filter kernel errors
3. Look for recurring patterns that might indicate failing components

Case 3: Tracking Down Application Crashes

When applications crash:

1. Check application-specific logs (e.g., /var/log/apache2/error.log)
2. Look at /var/log/syslog around the time of the crash
3. Check system resource usage in /var/log/syslog for out-of-memory events

How to Create Custom Log Files, Parsers, and Analysis Scripts

Sometimes the default logging isn't enough. Here's how to customize:

Setting Up Custom Logs with rsyslog

Create a new logging rule:

# Create a new rsyslog configuration
sudo nano /etc/rsyslog.d/custom.conf

# Add a rule to log SSH attempts to a dedicated file
if $programname == 'sshd' then /var/log/ssh.log

Writing Shell Scripts for Log Analysis

Here's a simple script to check for failed SSH attempts:

#!/bin/bash
LOGFILE="/var/log/auth.log"
FAILURES=$(grep "Failed password" $LOGFILE | wc -l)
echo "Found $FAILURES failed password attempts"

Save this as check-ssh-fails.sh, make it executable with chmod +x check-ssh-fails.sh, and run it when needed.

Wrapping Up

Understanding Ubuntu's log files gives you X-ray vision into your system. From troubleshooting issues to monitoring security or simply learning more about how your system runs, logs are an essential resource.

FAQs

What's the difference between /var/log/syslog and /var/log/messages in Ubuntu?

In Ubuntu, /var/log/syslog contains general system messages, similar to what /var/log/messages would contain in other Linux distributions. Ubuntu typically doesn't use /var/log/messages by default, as it consolidates this information in the syslog file.

How long are logs kept in Ubuntu by default?

By default, Ubuntu's logrotate configuration typically keeps logs for 4 weeks before deleting them, though this varies by log type. Critical logs might be kept longer, while high-volume logs might rotate more quickly.

How can I check if someone has attempted to hack my Ubuntu server?

Check /var/log/auth.log for failed login attempts, especially multiple failures from the same IP address. Also look for unusual sudo usage patterns that might indicate someone gained unauthorized access.

What should I do if my log files are filling up my disk?

First, identify which logs are growing fastest using du -sh /var/log/*. Then adjust logrotate settings for those files to rotate more frequently or keep fewer old logs. For immediate relief, you can manually remove old rotated logs (files ending with numbers or dates).

How do I enable debug logging for a specific service?

Most services have a configuration file where you can set the log level. For instance, to increase Apache's logging verbosity, edit /etc/apache2/apache2.conf and change LogLevel warn to LogLevel debug, then restart the service.

Can I send Ubuntu logs to a remote monitoring system?

Yes, Ubuntu's rsyslog can forward logs to remote systems. For modern observability platforms like Last9, you'll typically install their agent, which collects logs and forwards them to the monitoring platform.

How do I find all errors in my logs from the past hour?

Use a command like:

sudo find /var/log -type f -exec grep -l "error" {} \; | xargs grep "error" | grep "$(date +"%b %e %H" -d "1 hour ago")"

What's the best way to monitor logs in real-time on multiple servers?

For a small number of servers, you can use tail -f in multiple terminal windows. For larger deployments, set up a centralized logging system like rsyslog with Last9, ELK Stack, or Graylog to aggregate and monitor logs from all systems in one place.