# Traces Query Builder

> Visual query builder to filter, aggregate, and analyze traces without writing TraceQL

Source: https://last9.io/docs/traces-query-builder/

The Traces Query Builder provides a visual interface for constructing trace analysis queries through operation stages. Each stage represents a specific data manipulation function, allowing you to build complex queries without writing TraceQL syntax.

![Last9 Traces Query Builder in action](../../../../assets/content/docs/traces/traces-explorer/traces-query-builder.png)

## Accessing the Query Builder

Navigate to [Traces](https://app.last9.io/traces) in Last9 and use the **Span** or **Trace** tabs to begin building queries.

### Understanding Trace vs Span Mode

The Query Builder operates in two distinct modes that determine how results are displayed and what operations are available:

**Trace Mode** shows root spans of traces matching your query criteria:

- The results table displays one entry per trace (showing the root span)
- Clicking a result reveals the complete trace with all its spans
- Useful for understanding full request flows and overall trace characteristics
- Limited to FILTER operations only

**Span Mode** shows the exact spans matching your query criteria:

- The results table displays individual spans as separate entries
- Multiple spans from the same trace can appear as different rows
- Clicking a result shows a truncated view focused on the matching span, its children, and sibling spans from the same service
- Essential when you need to see specific spans in results (e.g., when querying particular span names or using negative filters)
- **Enables advanced query capabilities**: Click **ADD STAGE** to access AGGREGATE and TRANSFORM operations, create visualizations, save queries to dashboards, and set up streaming aggregations—all powered by TraceMetrics

:::note
When clicking on a result in Span mode, the duration and span count in the Trace Details sidepanel reflect the truncated trace view, not the full distributed trace. The TRACE ID column shows the trace identifier in both modes.
:::

## Core Stages

### Filter

The FILTER stage allows you to narrow down traces and spans based on specific conditions using various operators:

Supported Operators:

- **=**: Exact match
- **!=**: Not equal to
- **exists**: Field exists with any value
- **not exists**: Field does not exist
- **contains**: String contains substring
- **not contains**: String does not contain substring
- **matches**: Matches a pattern/regular expression

Parameters:

- **Field name**: The span or trace attribute to filter on
- **Operator**: One of the supported operators
- **Value**: The comparison value or pattern (not required for exists operators)

A single field and operator can have multiple values. These values are "ORed" together. To achieve AND logic, use multiple filter stages.

Examples:

```shell
Service Name exists
Trace Status = STATUS_CODE_ERROR
http.method != POST
span.name contains "database"
user.id matches "user-[0-9]+"
```

### Aggregate

The AGGREGATE stage supports various statistical computations with optional grouping and timeslicing:

1. Zero or One Argument Functions:

   - **count**
     - Usage without argument: `count as total_count`
     - Usage with argument: `count field_name as field_count`

2. Single Argument Functions:

   - **sum**: Calculate sum of values
   - **min**: Find minimum value
   - **max**: Find maximum value
   - **avg**: Calculate average
   - **median**: Find median value
   - **stddev**: Calculate sample standard deviation
   - **stddev_pop**: Calculate population standard deviation
   - **variance**: Calculate sample variance
   - **variance_pop**: Calculate population variance

   - Usage Syntax: `function_name field as result_name`

3. Two Argument Functions:

   - **quantile**: Calculate approximate quantile (value between 0 and 1)
   - **quantile_exact**: Calculate exact quantile (value between 0 and 1)

   - Usage: `quantile(0.99, duration) as p99_duration`

Additional Features:

- **groupby**: Group results by specified fields, similar to SQL GROUP BY
  - Example: `groupby Service Name as service_name`
- **timeslice**: Define time bucket intervals for time-series aggregation
  - When specified, creates time-series visualization
  - Without timeslice, displays only table view
  - Example: `timeslice 15 minutes`

**Example Query:**

```shell
count as _count
groupby Service Name as service_name
timeslice 15 minutes
```

### Transform

The TRANSFORM stage provides data transformation methods for manipulating span and trace attributes:

1. **split**

   - Parameters:

     - **from**: Source field
     - **on**: Delimiter character
     - **select part**: Part number to select (0-indexed)
     - **as**: Output field name

   - Example:

     ```sql
     split from http.url on / select part 2 as endpoint
     ```

2. **splitInto**

   - Parameters:

     - **from**: Source field
     - **on**: Delimiter character
     - **into**: Comma-separated list of output field names

   - Example:

     ```sql
     splitInto from trace.id on - into region, service, request_id
     ```

3. **replaceRegex**

   - Parameters:

     - **from**: Source field
     - **pattern**: Regex pattern to match
     - **with**: Replacement string
     - **as**: Output field name

   - Example:

     ```sql
     replaceRegex from user.email /[<>]/ with - as sanitized_email
     ```

4. **concat**

   - Parameters:

     - **from**: First field to concatenate
     - Additional fields/literals to concatenate
     - **as**: Output field name

   - Example:

     ```sql
     concat http.method, ' ', http.route as full_endpoint
     ```

5. **join**

   - Parameters:

     - **field**: Field to join
     - **separator**: String to use between values
     - **as**: Output field name

   - Example:

     ```sql
     join error.type , as error_types
     ```

6. **if**

   - Parameters:

     - **Condition**: `isEqual`, `!isEqual`, `isEmpty`, `!isEmpty`
     - **then**: Value if condition is true
     - **else**: Value if condition is false
     - **as**: Output field name

   - Example:

     ```sql
     if isEmpty error.message then 'No Error' else error.message as error_display
     ```

## Visualization Modes

The Query Builder provides two visualization modes:

### Visualization (Chart)

Time-series chart visualization is displayed when your query includes a **timeslice** parameter in the AGGREGATE stage. This mode shows trends over time and is ideal for monitoring patterns and anomalies.

![Query Builder Chart](../../../../assets/content/docs/traces/traces-explorer/query-builder-chart.png)

### Table

Table view displays aggregated results in tabular format. This is the only view available when no timeslice is specified, or you can switch to it manually when both modes are available.

![Query Builder Table](../../../../assets/content/docs/traces/traces-explorer/query-builder-table.png)

## Query Construction Best Practices

1. **Operation Order**

   - Start with FILTER operations to reduce data volume early
   - Apply TRANSFORM operations to prepare fields for aggregation
   - Use AGGREGATE operations last to summarize data
   - Apply additional FILTER stages after AGGREGATE to filter aggregated results

2. **Field Naming**

   - Use descriptive names for output fields in `as` clauses
   - Maintain consistent naming conventions (snake_case recommended)
   - Avoid special characters in field names
   - Use meaningful prefixes for grouped fields

3. **Performance Optimization**

   - Keep time ranges reasonable (avoid queries spanning months)
   - Filter on indexed fields like Service Name for faster execution
   - Use `exists` operator for field presence checks rather than string matching
   - Filter early in the query chain to minimize data processing
   - Use `=` operator on Service Name for significantly faster query execution

4. **Timeslice Guidelines**

   - Match timeslice intervals to your time range:
     - 1-hour range: 1-5 minute intervals
     - 1-day range: 5-30 minute intervals
     - 1-week range: 1-4 hour intervals
   - Shorter intervals provide more detail but slower queries
   - Longer intervals are faster but may miss short-lived patterns

5. **Error Prevention**

   - Use `exists` checks before accessing optional fields
   - Validate regex patterns before using them (tools like [regex101.com](https://regex101.com/))
   - Test queries on shorter time ranges before expanding
   - Use `!isEmpty` checks before transformations
   - When filtering by status (e.g., `Trace Status = STATUS_CODE_ERROR`), remember that filters apply at the span level in both modes - you may see traces marked "OK" that contain error spans

## Using Query Builder with Streaming Aggregation

Queries built in the Query Builder can be saved as Streaming Aggregation rules to generate TraceMetrics and continuously compute metrics from your trace data. This is particularly useful for creating custom metrics without impacting query performance.

     1. Build your query in the Query Builder with FILTER and AGGREGATE stages
     2. Click **Create Metric** to open the Streaming Aggregation configuration
     3. Configure the evaluation frequency (timeslice) and provide a descriptive rule name
     4. Save the rule to begin generating metrics

![Creating a streaming aggregation rule from Query Builder](../../../../assets/content/docs/traces/traces-explorer/traces-streaming-aggregation.png)

Your saved query will now run continuously, creating metrics you can query without processing raw trace data. Learn more about [Streaming Aggregation](/docs/streaming-aggregations/).

## Field Handling

### Common Trace Attributes

Access standard trace and span attributes using their exact names:

```
Service Name
Trace Status
http.method
http.status_code
http.route
error.type
error.message
duration
```

### Custom Attributes

Access custom attributes and tags added by your instrumentation:

```
user.id
tenant.id
feature.flag
environment
```

### Nested Fields

For nested attributes, use dot notation:

```
http.request.headers.user_agent
resource.attributes.service.version
```

## Keyboard Shortcuts

The Query Builder supports keyboard shortcuts for faster query construction:

- **Shift + Enter**: Add a new stage
- **Shift + Delete**: Remove the current stage
- **Tab**: Navigate between fields in a stage

:::note
Switching between Builder and Editor modes preserves both query states independently. You can switch modes without losing your work in either interface.
:::

---

## Troubleshooting

Please get in touch with us on [Discord](https://discord.com/invite/Q3p2EEucx9) or [Email](mailto:support@last9.io) if you have any questions.