Grafana Loki in Last9
Use Last9's embedded Grafana Loki to view logs.
Using Grafana Loki
Last9 provides a Grafana Loki interface using LogQL to explore your logs data.
- Access the Loki UI by visiting Explore and selecting Loki as the datasource.
- You can perform LogQL queries to explore logs in this interface. This is useful for structured exploration of logs data for people who are familiar with Grafana and Loki.
Note: You can also use Editor Mode to perform LogQL queries in Last9 Native Log Explorer as well.
LogQL Compatibility
Following functions in LogQL are supported:
RATE
COUNT_OVER_TIME
SUM_OVER_TIME
AVG_OVER_TIME
MAX_OVER_TIME
MIN_OVER_TIME
SUM
AVG
COUNT
MAX
MIN
STDDEV
MEDIAN
STDVAR
Following parsers in LogQL are supported:
json
regexp
Read more about the documentation for each function here.
Creating Dashboards
Accessing Grafana
- Navigate to the Grafana section in Last9
- Create a new dashboard by clicking Create Dashboard
- Add a new panel to begin visualizing your data
Selecting Loki Data Source
The Loki data source comes pre-configured in Last9’s embedded Grafana, so you can start querying immediately.
Query Construction Methods
Using Builder Mode
Builder mode provides a visual interface for constructing Loki queries without writing LogQL. Here’s how to use it:
-
Label Selection
- Click Add label to start building your query
- Select labels (e.g., service, severity) from the dropdown
- Choose operators (=, !=, =
, !) - Select or type values for the labels
-
Operations
- Add operations using the Operations button
- Common operations include:
- Line contains
- Line does not contain
- Line contains regex
- Line does not contain regex
- JSON
-
Aggregations
- Click Add range function
- Select functions like:
- Rate
- Count over time
- Sum over time
- Avg over time
- Set time windows ([1m], [5m], [1h])
-
Examples Using Builder Mode:
Basic Query:
- Label:
service = "auth-service"
- Operation:
Line contains "error"
- Range:
count_over_time [5m]
Advanced Query:
- Label:
service =~ "api.*"
- Label:
severity = "error"
- Operation:
JSON
- Operation:
Line contains "timeout"
- Range:
sum by (status_code)
- Label:
-
Builder to Code Mode
- Switch between modes to see the LogQL equivalent
- Learn LogQL syntax through the Builder interface
- Fine-tune queries in Code mode
Writing LogQL Queries
For advanced users or complex queries, you can write LogQL directly:
Basic Query Structure:
{service="your-service"}
Common Aggregation Patterns:
sum by (severity) (count_over_time({service="your-service"}[5m]))
Key Query Components
- Label matchers:
{label="value"}
- Line filters:
|= "error"
- Aggregation functions:
sum
,avg
,max
- Time windows:
[1m]
,[1h]
,[1d]
Understanding Window Behavior
Remember that Last9’s window behavior differs from standard Loki:
- Last9 uses tumbling windows (window size = step size)
- Both window and step size are defined by the
[]
parameter - For instant queries, match time range to window size
Creating Visualizations
Panel Types
-
Time Series
- Best for tracking metrics over time
- Suitable for rate and count queries
-
Bar Charts
- Good for comparing values across categories
- Works well with
sum by
aggregations
-
Tables
- Useful for detailed log analysis
- Can show multiple columns of log data
Panel Configuration
- Set appropriate panel title and description
- Configure axes and legends
- Set up thresholds and alerts if needed
- Choose color scheme for better visibility
Advanced Query Techniques
Using Multiple Queries
sum(rate({service="auth-service"} |= "error" [5m])) by (severity)sum(rate({service="auth-service"} |= "warning" [5m])) by (severity)
Pattern Matching
{service=~"auth.*"} |= "error" != "timeout"
Metric Extraction
sum by (status_code) (count_over_time({service="api"} | json | status_code != "" [5m]))
Dashboard Organization
Best Practices
- Group related panels logically
- Use consistent time ranges across related panels
- Add descriptive titles and documentation
- Consider user permissions and sharing settings
Layout Tips
- Arrange panels in order of importance
- Use rows to group related visualizations
- Consider different screen sizes and resolutions
Performance Optimization
Query Efficiency
- Use label filters before line filters
- Start with Service and Severity filters for better performance
- Avoid processing unnecessary data
Time Range Considerations
- Start with smaller time ranges during development
- Consider data retention policies
- Use appropriate aggregation intervals
Troubleshooting
Please get in touch with us on Discord or Email if you have any questions.