SSHD Logs 101: Configuration, Security, and Troubleshooting Scenarios

Secure Shell (SSH) is a fundamental tool for remote system administration, and its logs play a critical role in security monitoring, debugging, and compliance.

SSHD logs provide insights into authentication attempts, connection successes, failures, and potential intrusions.

This guide explores everything you need to know about SSHD logs, including their location, format, analysis, and lesser-known security practices to maximize their effectiveness.

What Are SSHD Logs?

SSHD logs are records generated by the SSH daemon (sshd), capturing authentication attempts, session activities, and security-related events. These logs help administrators track user logins, detect brute-force attacks, and troubleshoot connection issues.

Where Are SSHD Logs Stored?

The location of SSHD logs depends on the operating system and log configuration:

• Linux (Debian/Ubuntu): /var/log/auth.log
• Linux (RHEL/CentOS): /var/log/secure
• macOS: /var/log/system.log
• FreeBSD: /var/log/auth.log
• Custom Logging (Configured in /etc/ssh/sshd_config)

To verify the logging location, check the SSH configuration:

sudo grep -i log /etc/ssh/sshd_config

What does SSHD Log Entries Contain

Each SSHD log entry contains timestamps, process names, PIDs, IP addresses, and authentication statuses. A typical log entry looks like this:

Jan 30 12:45:23 server sshd[1234]: Accepted password for user1 from 192.168.1.100 port 54321 ssh2

Common Log Messages:

• Successful login: Accepted password for <user>
• Failed login: Failed password for <user>
• Public key authentication: Accepted publickey for <user>
• Brute-force attempt: Maximum authentication attempts exceeded
• Disconnection: Disconnected from <IP> port <port>

How to Analyze SSHD Logs

Using grep to Filter Logs

Extract authentication failures:

grep "Failed password" /var/log/auth.log

Find all login attempts from a specific IP:

grep "192.168.1.100" /var/log/auth.log

Identify root login attempts:

grep "root" /var/log/auth.log

Using journalctl for Systemd Logs

On modern Linux systems using systemd:

journalctl -u sshd --since "1 hour ago"

Visualizing SSHD Logs

For a better overview, you can use tools like goaccess or fail2ban to analyze SSH logs.

grep "sshd" /var/log/auth.log | goaccess --log-format=COMBINED

How to Configure sshd Logging

Logging is a crucial part of managing SSH (sshd), helping track access, troubleshoot issues, and enhance security.

The sshd daemon allows the configuration of logging settings via the sshd_config file. Two key options control how logs are recorded:

Adjusting LogLevel

The LogLevel directive determines the verbosity of SSH logs. It controls how much detail is logged, ranging from minimal messages to detailed debugging output. Common values include:

• QUIET – Logs almost nothing.
• FATAL – Only logs critical errors.
• ERROR – Logs errors that could impact functionality.
• INFO – Default setting; logs key connection events.
• VERBOSE – More detailed than INFO, useful for monitoring.
• DEBUG / DEBUG1-3 – Extensive logging, useful for troubleshooting but not recommended for production due to performance overhead.

To set the log level, edit /etc/ssh/sshd_config:

sudo nano /etc/ssh/sshd_config

Find or add the LogLevel directive:

LogLevel VERBOSE

Save the file and restart sshd to apply the changes:

sudo systemctl restart sshd

How to Configure Syslog Facility

The SyslogFacility directive defines which system logging facility sshd should use when sending logs to syslog. This allows SSH logs to be directed to specific log files based on your system's logging configuration.

Common values include:

• AUTH – Authentication-related logs (default).
• AUTHPRIV – Similar to AUTH, logs are restricted to privileged users.
• DAEMON – Logs are categorized under system daemons.
• USER – Logs are recorded under user-level messages.

To configure it, edit /etc/ssh/sshd_config:

SyslogFacility AUTHPRIV

After making changes, restart sshd:

sudo systemctl restart sshd

Where to Find SSH Logs

• On most Linux systems, logs are stored in /var/log/auth.log (Debian/Ubuntu) or /var/log/secure (RHEL/CentOS).
• Use journalctl -u sshd for system-based logs.

Properly configuring sshd logging helps with monitoring and security, ensuring you have the right level of visibility into SSH activity.

How to Improve SSHD Log Security

Enable Verbose Logging

Increase the logging level in /etc/ssh/sshd_config for more detailed logs:

LogLevel VERBOSE

Restart the SSH service to apply changes:

sudo systemctl restart sshd

Monitor Logs with Fail2Ban

Fail2Ban automatically blocks IPs with excessive failed login attempts:

sudo apt install fail2ban
sudo systemctl enable --now fail2ban

Use SSHD Audit Logs for Advanced Tracking

Enable audit logging for SSH:

auditctl -w /etc/ssh/sshd_config -p wa

Check audit logs:

aureport --start today --failed

Analyzing and Resolving SSHD Log Issues

When SSH connections go awry, the first step in fixing them often involves diving into the logs. One powerful way to track down what’s happening is by increasing the verbosity of your SSHD logs.

This lets you see more detailed information about the connection process, which can help pinpoint where things are breaking down.

Increasing Log Verbosity

By default, SSHD logs only the most critical information, which can be a bit vague. To get more insight, you can adjust the log level to show more detailed data. To do this, you’ll need to modify the SSHD configuration file (/etc/ssh/sshd_config).

1. Edit the SSHD Config File
   Open the config file in a text editor:

sudo nano /etc/ssh/sshd_config

2. Set LogLevel to DEBUG
   Find the LogLevel directive in the file. If it’s commented out, uncomment it, and set it to DEBUG:

LogLevel DEBUG

• The DEBUG level will give you a detailed account of each step the server takes to process a connection, which is helpful for troubleshooting.

3. Restart SSHD
   After saving the changes, restart SSHD to apply the new logging settings:

sudo systemctl restart sshd

Analyzing SSHD Logs

Once you've bumped up the verbosity, the logs will show up in /var/log/auth.log (or a similar location, depending on your system). This is where you’ll find the juicy details about failed logins, key exchanges, authentication attempts, and other critical events.

Look for lines like these:

• Authentication failures:

sshd[12345]: Failed password for user1 from 192.168.1.100 port 22 ssh2

• Successful connections:

sshd[12345]: Accepted password for user1 from 192.168.1.100 port 22 ssh2

• Key exchange info:

sshd[12345]: key exchange finished: algorithm: diffie-hellman-group14-sha1, server host key algorithm: ssh-rsa

Common Troubleshooting Steps

• Check for incorrect permissions:
  SSH can fail if the file or directory permissions are too open. Ensure that the .ssh directory and authorized keys file have the correct permissions:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

• Verify the firewall:
  Sometimes, network issues can cause SSH failures. Double-check if any firewalls or security groups are blocking port 22 or your custom SSH port.
• Examine the authentication methods:
  If you're seeing Failed password messages, the issue might be with the credentials or authentication method. Check if password authentication is enabled in the SSHD config file (PasswordAuthentication yes).

Best Practices for SSHD Logging

Detect Hidden Tunnels via SSH

Attackers sometimes use SSH for unauthorized tunnels. Detect them:

netstat -tnp | grep sshd

Track User Sessions in Real-Time

Monitor active SSH sessions:

w
who

Log and Alert Unusual SSH Activities

Set up real-time alerts for suspicious logins:

tail -f /var/log/auth.log | grep "Failed password" | while read line; do echo "$line" | mail -s "SSH Alert" admin@example.com; done

Conclusion

SSHD logs are a goldmine for security and operational insights. Regular log analysis helps identify brute-force attacks, unauthorized access, and system misconfiguration.