Syslog is a foundational part of logging in Linux and Unix-based systems, helping engineers efficiently capture and analyze system events.
Among its core components, syslog levels play a crucial role in categorizing logs based on their severity. Understanding these levels can significantly improve troubleshooting, monitoring, and alerting strategies.
What Are Syslog Levels?
Syslog levels define the priority of a log message, helping systems and administrators determine which logs need immediate attention and which can be reviewed later.
The syslog protocol (as defined in RFC 5424) standardizes these levels, assigning each a numerical value from 0 (Emergency) to 7 (Debug).
The 8 Syslog Severity Levels (From Most Critical to Least)
- Emergency (0): System is unusable (e.g., kernel panic, total hardware failure).
- Alert (1): Immediate action required (e.g., database corruption, security breach).
- Critical (2): Critical conditions that could lead to failure (e.g., disk errors, power issues).
- Error (3): Non-critical errors but issues that need fixing (e.g., failed service restart).
- Warning (4): Something isn't right but not necessarily an error (e.g., high memory usage).
- Notice (5): Normal but noteworthy events (e.g., service restart, configuration changes).
- Informational (6): General system activity logs (e.g., user logins, service status updates).
- Debug (7): Detailed information for troubleshooting (e.g., function calls, debug traces).
Each level serves a distinct purpose, allowing administrators to filter and analyze logs effectively.
Why Syslog Levels Matter
Efficient Log Filtering and Analysis
Syslog levels help engineers prioritize logs based on their importance. Instead of sifting through thousands of lines, logs can be filtered to focus on critical alerts or error messages.
Automation and Alerting
Many monitoring tools, such as Last9, Prometheus, or Datadog, integrate with syslog and use severity levels to trigger alerts. For instance, an Alert (1) might immediately notify an on-call engineer, while Informational (6) logs might just be stored for audit purposes.
Optimizing Storage and Retention
Logging everything indefinitely can be expensive. By categorizing logs, organizations can set retention policies—keeping critical logs longer while purging debug logs more frequently.
Step-by-Step Implementation of Syslog Levels in Network Devices
Network devices—such as routers, switches, and firewalls—generate syslog messages to report events ranging from critical failures to routine operations. Properly implementing syslog levels in these devices ensures efficient monitoring and troubleshooting.
1. Configuring Syslog on Network Devices
Most network devices support syslog logging by default but require configuration to direct logs to a central syslog server. Configuration typically involves:
- Enabling Syslog: Activating logging services on the device.
- Setting Severity Levels: Defining which log levels to capture (e.g., only errors and higher).
- Specifying Syslog Server: Directing logs to an external syslog collector for storage and analysis.
Example configuration for a Cisco router:
logging on
logging host 192.168.1.100
logging trap warnings
This sends log messages of severity Warning (Level 4) and higher to the syslog server at 192.168.1.100.
2. Technical Considerations
Log Volume Management
High-frequency events like interface state changes can generate excessive logs. To prevent overload:
- Use rate-limiting to restrict log frequency.
- Set logging levels to focus on critical events in production.
Log Formatting & Structure
Different devices may format logs differently. Using structured logging (e.g., JSON) simplifies parsing:
{
"timestamp": "2025-01-29T12:34:56Z",
"device": "Router-1",
"severity": "error",
"message": "BGP session down"
}
Reliable Log Transmission
Syslog messages can be sent via:
- UDP (Default, Port 514): Faster but lacks reliability.
- TCP (Port 1468 or custom): Ensures log delivery with acknowledgments.
- TLS Encryption: Protects log data from interception.
Integration with Monitoring & SIEM Tools
Forwarding syslog messages to SIEM (Security Information and Event Management) platforms like ELK stack enhances visibility and enables real-time analysis.
- Best Practices for Configuring Syslog Levels
- Filter and Route Logs Effectively: Ensure high-severity logs trigger alerts, while lower-level logs are archived for later analysis.
- Use Structured Logging: Adding metadata (like request ID or hostname) helps in tracing and correlating issues.
- Adjust Logging Based on Environment: Production logs should prioritize higher severity levels, while development logs can include debug details.
3 Interesting Aspects of Syslog Levels
1. Facility Codes: The Other Half of Syslog Messages
Syslog messages aren’t just about severity; they also have facility codes, which indicate where the log originated. Common facilities include:
- auth (4) - Authentication logs
- cron (9) - Scheduled jobs
- daemon (3) - System daemons
- local0-7 (16-23) - Custom applications
Pairing facility codes with severity levels allows for
2. Not All Systems Log the Same Way
Different operating systems and services implement syslog levels in their way. For example:
- Cisco devices treat level 0-2 logs as immediate threats.
- Apache/Nginx may log at Debug (7) but require additional flags to show verbose details.
- Cloud services like AWS CloudWatch often map syslog levels into structured JSON logs.
3. Some Applications Misuse Log Levels
While syslog levels are standardized, some applications mislabel logs—e.g., marking an informational event as an error or vice versa. This can lead to false positives or missed issues.
Best Practices for Managing Syslog Levels
1. Set Up Proper Log Filtering
Use tools like rsyslog or syslog-ng to filter logs efficiently. Example:
if $syslogseverity <= 3 then /var/log/critical.log
This rule captures all logs with severity 3 (Error) or higher into a dedicated file.
2. Implement Centralized Logging
Instead of managing logs on individual servers, centralized logging solutions like Last9, ELK (Elasticsearch, Logstash, Kibana), or Loki can help analyze logs across multiple machines.
3. Use Log Rotation to Prevent Overflow
Large log files slow down performance. Configure logrotate to keep logs manageable:
/var/log/syslog { weekly rotate 4 compress }
This keeps syslog files for four weeks before compressing old logs.
4. Correlate Logs with Metrics
Logs alone don't always show the full picture. Correlating syslog data with CPU, memory, and network metrics in platforms like Last9 or Grafana can help pinpoint root causes faster.
Application of Syslog Levels
Syslog severity levels help categorize log messages based on urgency, guiding teams in monitoring and troubleshooting. Here’s how they apply in real-world scenarios, along with examples and best practices.
1. Emergency (Level 0) – System is Unusable
Example: A critical hardware failure causing a full system crash.
Best Practice: Configure alerts to immediately notify on-call engineers and trigger automated recovery actions where possible.
2. Alert (Level 1) – Immediate Action Required
Example: Database corruption detected, risking data integrity.
Best Practice: Route these logs to a dedicated alerting system that ensures immediate response.
3. Critical (Level 2) – Critical Conditions
Example: Application running out of memory, leading to crashes.
Best Practice: Set up threshold-based monitoring to detect and address issues before they escalate.
4. Error (Level 3) – Errors that Need Attention
Example: Authentication failures due to incorrect credentials or expired tokens.
Best Practice: Log errors with enough context to aid debugging, while avoiding sensitive data exposure.
5. Warning (Level 4) – Potential Issues
Example: High CPU usage nearing system limits but still operational.
Best Practice: Use warnings to track trends that might lead to future failures, integrating them into dashboards for proactive monitoring.
6. Notice (Level 5) – Normal but Significant Events
Example: Configuration changes such as enabling a new feature flag.
Best Practice: Store notices for audit trails and change tracking without flooding alert systems.
7. Informational (Level 6) – General Information
Example: User logins or scheduled backups are completed successfully.
Best Practice: Keep informational logs accessible for analysis but avoid cluttering real-time monitoring dashboards.
8. Debug (Level 7) – Debugging Messages
Example: Function execution traces for diagnosing complex issues.
Best Practice: Enable debug logging selectively in development and staging environments to prevent excessive log volume in production.
Difference between Syslog Facilities and Severity Levels
Syslog messages are categorized using facilities and severity levels, allowing for structured logging and easier filtering. While severity levels indicate the urgency of an event, facilities define the source or origin of the log message.
Understanding Syslog Facilities
Facilities group log messages based on their source, such as system processes, authentication events, or mail services. Each facility has a unique numeric code.
Facility Code | Facility Name | Description |
---|---|---|
0 | Kernel | Messages from the operating system kernel |
1 | User-Level | Application-level logs from user processes |
2 | Email-related logs | |
3 | System Daemons | Logs from background system services |
4 | Authorization | Authentication and security-related messages |
5 | Syslog Daemon | Logs from the syslog process itself |
6 | Line Printer | Printer-related messages |
7 | Network News | Logs related to news or messaging services |
8 | UUCP | Unix-to-Unix Copy Protocol logs |
9 | Clock Daemon | Time synchronization messages |
10 | AuthPriv | Security and authentication logs with restricted access |
11 | FTP | File Transfer Protocol logs |
16-23 | Local Use (0-7) | Custom logs for application-specific use |
How Facilities and Severity Levels Interact
Each syslog message has both a facility and a severity level, forming a structured log entry. The priority (PRI) value is calculated using the following formula:
PRI = (Facility × 8) + Severity
For example, an authentication failure (Facility 4, Severity 3 - Error) would have:
(4 × 8) + 3 = 35
This priority value helps syslog servers process and filter logs efficiently.
Practical Use in Logging Configuration
- Filtering Logs: Administrators can configure syslog servers to collect only specific facilities and severities, reducing noise.
- Routing Logs: Different log sources (e.g., authentication, networking) can be directed to separate files or alerting systems.
- Security Considerations: Critical logs like authpriv should be stored securely and restricted from unauthorized access.
A Quick Reference to Syslog Severity Levels
The table below references syslog severity levels, their numerical values, descriptions, and typical use cases.
Level | Severity | Description | Example Use Case |
---|---|---|---|
0 | Emergency | System is unusable | Hardware failure causing a total system crash |
1 | Alert | Immediate action required | Database corruption detected |
2 | Critical | Critical conditions | Firewall dropping all incoming traffic unexpectedly |
3 | Error | Errors that need attention | Authentication failures due to invalid credentials |
4 | Warning | Potential issues | High CPU usage nearing threshold |
5 | Notice | Normal but significant events | Configuration changes applied to a router |
6 | Informational | General operational messages | User logins or scheduled backups completed |
7 | Debug | Debugging messages for developers | Function execution traces for troubleshooting |
Wrapping Up
Syslog levels are fundamental for structured logging, helping engineers diagnose issues efficiently.
Beyond just knowing the levels, understanding how different applications log, configuring filtering rules, and integrating logs with metrics can make troubleshooting significantly easier.
FAQs About Syslog Levels
1. What are syslog severity levels?
Syslog severity levels categorize log messages based on urgency, ranging from Emergency (0) for system failures to Debug (7) for detailed troubleshooting messages.
2. Why are syslog levels important?
They help teams prioritize issues, filter logs efficiently, and ensure critical events get immediate attention while routine messages remain accessible for analysis.
3. How many syslog severity levels exist?
There are eight levels, numbered from 0 (Emergency) to 7 (Debug).
4. Can I configure which syslog levels to log?
Yes, most systems allow filtering logs by severity. For example, you can log only warnings and above to reduce noise.
5. What’s the difference between syslog facilities and severity levels?
- Severity levels indicate the urgency of a log message.
- Facilities define the source of the log, such as authentication, networking, or system processes.
6. How do syslog levels affect monitoring and alerting?
Alerts are typically triggered by higher severity logs (Error and above), while lower severity logs (Notice and below) are stored for analysis and debugging.
7. What happens if all syslog messages are logged?
Logging everything, especially Debug messages, can generate excessive log volume, leading to storage issues and performance degradation. It’s best to log selectively based on the use case.
8. Do all devices and applications use the same syslog levels?
While syslog is standardized, different vendors may interpret severity levels slightly differently. Always check device or application documentation.
9. How can I ensure important syslog messages are not missed?
- Set up log filtering to capture relevant severity levels.
- Use centralized logging to aggregate logs from multiple sources.
- Implement alerts for critical events.
10. Is syslog used only for networking devices?
No, syslog is widely used across servers, applications, and security systems to capture and manage logs efficiently.