Understanding Syslog Formats: A Quick and Easy Guide

Syslog is the backbone of logging in many Linux and Unix-based systems, playing a crucial role in monitoring, debugging, and auditing. But not all syslog messages are created equal.

Depending on your system, software, and logging configuration, syslog messages may follow different formats. This guide walks you through the different syslog formats, why they matter, and how to work with them effectively.

What Is Syslog?

Syslog is a standard protocol for message logging that allows applications and systems to send event data to a centralized location.

It helps in tracking system activity, diagnosing problems, and ensuring compliance with security policies. Syslog messages typically include timestamps, severity levels, source application information, and the actual log message.

The Core Functionality of Syslog

Syslog operates through a client-server architecture, where applications generate log messages and send them to a central syslog server. The core processes involved include:

• Message Generation: Applications, system services, and network devices generate log messages based on events.
• Message Transmission: Logs are sent over UDP (default) or TCP, optionally encrypted using TLS for security.
• Message Processing: The syslog server receives, filters, and stores logs based on predefined rules.
• Storage & Forwarding: Logs are written to files, and databases, or forwarded to other logging systems for further analysis.

This architecture makes syslog a scalable and flexible solution for system monitoring and troubleshooting.

What Are the Different Syslog Security Levels

Syslog categorizes messages into different security levels to help prioritize and manage system events. These levels, defined by their severity, range from emergencies requiring immediate attention to purely informational messages.

• Emergency (0): System is unusable (e.g., kernel panic).
• Alert (1): Action must be taken immediately (e.g., database corruption detected).
• Critical (2): Critical conditions (e.g., hardware failures).
• Error (3): General error messages.
• Warning (4): Potentially harmful situations.
• Notice (5): Normal but significant events.
• Informational (6): General informational messages.
• Debug (7): Debugging-level messages for troubleshooting.

What Are the Core Components of a Syslog Server and Message?

A syslog server consists of several key components that work together to collect, process, and store log messages:

• Syslog Listener: Listens for incoming log messages over UDP, TCP, or TLS.
• Message Parser: Extracts and categorizes log details like timestamps, priority, and source.
• Storage System: Saves logs in files, databases, or cloud-based storage.
• Filtering & Processing Engine: Applies rules to filter, forward, or trigger alerts based on log content.
• Visualization & Reporting Tools: Interfaces like Kibana, Grafana, or dedicated dashboards provide insights into log data.

Each of these components plays a crucial role in ensuring reliable and efficient log management in an infrastructure.

Key Components of a Syslog Message

Each of these components plays a crucial role in ensuring reliable and efficient log management in an infrastructure. Syslog messages themselves are composed of several key elements that contribute to their functionality:

• Priority (PRI): Encodes both facility and severity levels.
• Timestamp: Indicates when the log message was generated.
• Hostname: Identifies the source system of the log.
• Application Name: Specifies which process or service generated the log.
• Process ID (PID): Helps distinguish between different instances of the same application.
• Message Content: The actual event or status information being logged.

What is the Syslog Protocol and How Does It Work?

The Syslog protocol provides a standardized way for devices and applications to send log messages to a centralized server. It operates over UDP (port 514) by default, though TCP and TLS are also supported for reliability and security.

Syslog messages follow a structured format that typically includes a priority value, severity levels, timestamps, hostnames, application names, and the actual log message.

This format allows the protocol to be flexible in collecting and filtering log data, making it ideal for monitoring system performance, ensuring security, and assisting with compliance auditing.

How Syslog Works:

1. Device or Application Sends a Log: Devices, servers, or applications generate log messages that contain information about their activities (e.g., error messages, status reports, or security alerts).
2. Log Transmission: These messages are transmitted using the Syslog protocol to a centralized Syslog server, typically over UDP. However, TCP or TLS can be used to ensure more reliable message delivery and security.
3. Log Reception and Parsing: Once the Syslog server receives the log messages, they are parsed and stored in a structured format. Tools like rsyslog, syslog-ng, or the native system logging daemon process these logs.
4. Log Storage and Forwarding: Syslog messages are stored locally or forwarded to other systems for further processing, analysis, or long-term storage.
5. Log Analysis: Security analysts, system administrators, and monitoring tools can use the logs to troubleshoot issues, monitor system health, and identify potential security threats.

The Syslog protocol is widely supported by operating systems, networking devices, and security appliances. It remains a crucial component of modern IT infrastructure due to its simplicity and versatility in log management.

Key Syslog Formats and Standards to Know

1. BSD Syslog Format

One of the oldest and most widely used formats, the BSD syslog format consists of three main parts:

<PRI> TIMESTAMP HOSTNAME APPLICATION[PID]: MESSAGE

• <PRI> → Priority, calculated from facility and severity levels.
• TIMESTAMP → Date and time of the event.
• HOSTNAME → System where the message originated.
• APPLICATION[PID] → Application name and process ID.
• MESSAGE → The actual log message.

Example:

<34>Feb 14 12:34:56 my-server sshd[1234]: Failed password for user root

2. IETF Syslog Format (RFC 5424)

To overcome BSD format limitations, the IETF introduced RFC 5424, which provides more structure. Its format is:

<PRI>VERSION TIMESTAMP HOSTNAME APP-NAME PROCID MSGID STRUCTURED-DATA MESSAGE

• VERSION → Syslog protocol version.
• MSGID → Unique identifier for the message type.
• STRUCTURED-DATA → Optional key-value pairs for additional metadata.

Example:

<34>1 2025-02-14T12:34:56Z my-server sshd 1234 ID47 - Failed password for user root

3. JSON Syslog Format

Modern applications often use JSON-based logging for easier parsing and richer context.

Example:

{
  "severity": "error",
  "timestamp": "2025-02-14T12:34:56Z",
  "hostname": "my-server",
  "application": "sshd",
  "pid": 1234,
  "message": "Failed password for user root"
}

JSON-based logging integrates well with log management systems like ELK Stack, Last9, and cloud-based observability platforms.

Why Do Syslog Formats Matter?

Understanding syslog formats is essential for:

• Efficient log parsing → Tools like rsyslog and syslog-ng use format-specific rules to extract valuable data.
• Security and compliance → Regulations often require structured logging to track access attempts and system changes.
• Seamless monitoring integration → Whether using Splunk, Datadog, or Last9, properly formatted logs improve indexing and query performance.

How to Configure Syslog Format in Rsyslog

Rsyslog is a highly flexible syslog daemon that supports multiple logging formats. To set a specific format, update the /etc/rsyslog.conf file.

Configuring Rsyslog for RFC 5424 Format

To enable RFC 5424-compliant logging, modify the configuration:

module(load="imuxsock")
module(load="imklog")
*.* action(type="omfile" file="/var/log/syslog" template="RSYSLOG_ForwardFormat")

Configuring Rsyslog for JSON Format

For structured logging in JSON, use:

$template JsonFormat,"{\"timestamp\":\"%timestamp:::date-rfc3339\",\"host\":\"%HOSTNAME%\",\"app\":\"%syslogtag%\",\"msg\":\"%msg%\"}\n"
*.* action(type="omfile" file="/var/log/syslog" template="JsonFormat")

After making changes, restart rsyslog:

systemctl restart rsyslog

To convert BSD syslog messages to JSON and send them to Last9, you can use OpenTelemetry Collector with the syslog receiver and file exporter. This approach ensures structured logging and seamless integration with Last9's observability platform.

Converting BSD Syslog to JSON with OpenTelemetry and Last9

1. Configure OpenTelemetry Collector to Ingest and Transform Logs

Create a configuration file (otel-collector-config.yaml) for OpenTelemetry Collector:

receivers:
  syslog:
    tcp:
      listen_address: 0.0.0.0:514
    protocol: rfc5424

processors:
  transform:
    log_statements:
      - context: log
        statements:
          - set(attributes["timestamp"], format_timestamp(attributes["timestamp"], "rfc3339"))
          - set(attributes["log.level"], attributes["severity"])
          - rename(attributes["hostname"], "host")
          - rename(attributes["app"], "application")
          - rename(attributes["msg"], "message")

exporters:
  otlphttp:
    endpoint: "https://api.last9.io/v1/logs"
    headers:
      authorization: "Bearer <YOUR_LAST9_API_KEY>"

service:
  pipelines:
    logs:
      receivers: [syslog]
      processors: [transform]
      exporters: [otlphttp]

2. Deploy the OpenTelemetry Collector

Run the OpenTelemetry Collector with this configuration:

otelcol --config otel-collector-config.yaml

Why Use Last9 for Syslog Conversion?

• Structured Logging: Converts unstructured BSD syslog into a JSON-based format for better indexing.
• Seamless Observability: Direct integration with Last9 ensures logs are visualized alongside metrics and traces.
• Efficient Storage & Querying: Optimized for high cardinality log data without performance degradation.

Advantages and Disadvantages of Syslog

Advantages

• Centralized logging – Makes it easy to monitor multiple systems from a single place.
• Lightweight and efficient – Works well even in resource-constrained environments.
• Widely supported – Almost every OS and network device supports syslog.
• Flexible filtering and forwarding – Tools like rsyslog and syslog-ng let you manage logs efficiently.

Disadvantages

• No encryption by default – Traditional syslog doesn’t secure logs, so extra measures are needed.
• Limited structure in older formats – BSD syslog can be harder to parse compared to JSON or RFC 5424.
• Risk of log loss – If not configured properly, syslog messages might get dropped under heavy load.
• Troubleshooting challenges – Unstructured syslog data can be tough to analyze in large-scale environments.

Final Thoughts

Syslog might not be top of mind daily, but it plays a key role in organizing logs, improving troubleshooting, and strengthening security. Whether you’re using the classic BSD syslog, structured RFC 5424, or JSON-based logging, understanding these formats helps you manage infrastructure more effectively.

If you’re looking for a scalable, OpenTelemetry-native observability platform, Last9 simplifies telemetry data management, reduces downtime, and speeds up issue resolution. Book some time with us today or try for free!