# Logs Query API

> Use Logs Query API for searching and retrieving logs programmatically from your services.

Source: https://last9.io/docs/query-logs-api/

Last9 provides a powerful API for querying logs from your services. This document explains how to use the Logs Query API to search and retrieve logs programmatically.

:::tip
This API is for querying log data. For sending logs to Last9, see the [OpenTelemetry integration](/docs/integrations/observability/opentelemetry/).
:::

## Prerequisites

You can find necessary credentials on the [API Access](https://app.last9.io/settings/api-access) page. For detailed instructions on generating tokens, see [Getting Started with API](/docs/getting-started-with-api/).

### Base URL

```text
https://app.last9.io/api/v4/organizations/{org}
```

Replace `{org}` with your organization slug.

### Authentication

The API requires a Bearer token in the `X-LAST9-API-TOKEN` header.

```text
X-LAST9-API-TOKEN: Bearer <ACCESS_TOKEN>
```

See [Getting Started with API](/docs/getting-started-with-api/) for instructions on generating tokens.

---

## Query Logs

The primary endpoint for querying log data with pipeline-based filtering.

```text
POST /logs/api/v2/query_range/json
```

Full URL:

```text
https://app.last9.io/api/v4/organizations/{org}/logs/api/v2/query_range/json
```

### Query Parameters

| Parameter   | Type    | Required | Description                         |
| ----------- | ------- | -------- | ----------------------------------- |
| `start`     | integer | Yes      | Start time in Unix nanoseconds      |
| `end`       | integer | Yes      | End time in Unix nanoseconds        |
| `limit`     | integer | No       | Maximum number of logs to return    |
| `direction` | string  | No       | Sort order: `forward` or `backward` |
| `step`      | string  | No       | Time step for aggregations          |
| `offset`    | integer | No       | Pagination offset                   |
| `region`    | string  | No       | Cloud region to query logs from     |

### Request Body

The request body contains the JSON pipeline:

```json
{
  "pipeline": [
    {
      "type": "filter",
      "query": {
        "$and": [{ "$eq": ["service", "api-gateway"] }]
      }
    }
  ]
}
```

### Example Request

```shell
curl -X POST 'https://app.last9.io/api/v4/organizations/{org}/logs/api/v2/query_range/json?start=1743500000000000000&end=1743510000000000000&limit=100&direction=backward&region=ap-south-1' \
  -H 'X-LAST9-API-TOKEN: Bearer <ACCESS_TOKEN>' \
  -H 'Content-Type: application/json' \
  -d '{
    "pipeline": [
      {
        "type": "filter",
        "query": {
          "$and": [
            { "$eq": ["service", "api-gateway"] }
          ]
        }
      }
    ]
  }'
```

Replace `{org}` with your organization slug, `<ACCESS_TOKEN>` with your token, and `region` with your cloud region (e.g., `ap-south-1`).

### Response

```json
{
  "status": "success",
  "data": {
    "resultType": "streams",
    "result": [
      {
        "stream": {
          "service": "api-gateway",
          "level": "error",
          "env": "production"
        },
        "values": [
          ["1743505000000000000", "Connection timeout after 30s"],
          ["1743504990000000000", "Request failed: upstream unavailable"],
          ["1743504980000000000", "Error processing request"]
        ]
      }
    ],
    "stats": {
      "summary": {
        "bytesProcessedPerSecond": 1048576,
        "linesProcessedPerSecond": 500,
        "totalBytesProcessed": 2097152,
        "totalLinesProcessed": 1000,
        "execTime": 0.25
      }
    }
  }
}
```

---

## Discover Available Labels

Retrieve the list of labels available for filtering logs.

```text
GET /logs/api/v1/labels
```

### Query Parameters

| Parameter | Type    | Required | Description                    |
| --------- | ------- | -------- | ------------------------------ |
| `start`   | integer | Yes      | Start time in Unix nanoseconds |
| `end`     | integer | Yes      | End time in Unix nanoseconds   |

### Example Request

```shell
curl -X GET 'https://app.last9.io/api/v4/organizations/{org}/logs/api/v1/labels?start=1743000000000000000&end=1743600000000000000' \
  -H 'X-LAST9-API-TOKEN: Bearer <ACCESS_TOKEN>'
```

### Response

```json
{
  "status": "success",
  "data": ["service", "level", "env", "host", "namespace"]
}
```

---

## Discover Label Values

Retrieve possible values for a specific label.

```text
GET /logs/api/v1/label/{labelName}/values
```

### Path Parameters

| Parameter   | Type   | Description    |
| ----------- | ------ | -------------- |
| `labelName` | string | The label name |

### Query Parameters

| Parameter | Type    | Required | Description                    |
| --------- | ------- | -------- | ------------------------------ |
| `start`   | integer | Yes      | Start time in Unix nanoseconds |
| `end`     | integer | Yes      | End time in Unix nanoseconds   |

### Example Request

```shell
curl -X GET 'https://app.last9.io/api/v4/organizations/{org}/logs/api/v1/label/service/values?start=1743000000000000000&end=1743600000000000000' \
  -H 'X-LAST9-API-TOKEN: Bearer <ACCESS_TOKEN>'
```

### Response

```json
{
  "status": "success",
  "data": [
    "api-gateway",
    "user-service",
    "payment-service",
    "notification-service"
  ]
}
```

---

## Pipeline Syntax

The Logs API uses a JSON pipeline format for filtering log data. Multiple stages can be chained together.

### Filter Stage

Filter logs based on conditions:

```json
{
  "type": "filter",
  "query": {
    "$and": [{ "$eq": ["service", "api-gateway"] }]
  }
}
```

### Where Stage

Add additional filter conditions with OR/NOT logic:

```json
{
  "type": "where",
  "query": {
    "$or": [{ "$eq": ["level", "error"] }, { "$eq": ["level", "warn"] }]
  }
}
```

### Parse Stage

Extract fields from log body:

```json
{
  "type": "parse",
  "parser": "json",
  "field": "body",
  "labels": {
    "user_id": null,
    "request_id": null
  }
}
```

### Filter Operators

| Operator       | Description                   | Example                                  |
| -------------- | ----------------------------- | ---------------------------------------- |
| `$eq`          | Exact match                   | `{ "$eq": ["service", "api-gateway"] }`  |
| `$neq`         | Does not match                | `{ "$neq": ["level", "debug"] }`         |
| `$contains`    | Substring match               | `{ "$contains": ["body", "error"] }`     |
| `$notcontains` | Does not contain substring    | `{ "$notcontains": ["body", "test"] }`   |
| `$regex`       | Regular expression match      | `{ "$regex": ["path", "/api/v[0-9]+"] }` |
| `$notregex`    | Does not match regex          | `{ "$notregex": ["path", "health"] }`    |
| `$gt`          | Numeric greater than          | `{ "$gt": ["status_code", "400"] }`      |
| `$lt`          | Numeric less than             | `{ "$lt": ["status_code", "500"] }`      |
| `$gte`         | Numeric greater than or equal | `{ "$gte": ["duration", "1000"] }`       |
| `$lte`         | Numeric less than or equal    | `{ "$lte": ["duration", "5000"] }`       |

### Logical Operators

Combine multiple conditions using logical operators:

#### AND (All conditions must match)

```json
{
  "$and": [
    { "$eq": ["service", "api-gateway"] },
    { "$contains": ["body", "error"] }
  ]
}
```

#### OR (Any condition can match)

```json
{
  "$or": [{ "$eq": ["level", "error"] }, { "$eq": ["level", "fatal"] }]
}
```

#### NOT (Negate conditions)

```json
{
  "$not": [{ "$and": [{ "$eq": ["level", "debug"] }] }]
}
```

#### Combining Logical Operators

Nest logical operators to build complex queries:

```json
{
  "$and": [
    { "$eq": ["service", "api-gateway"] },
    { "$or": [{ "$eq": ["level", "error"] }, { "$eq": ["level", "fatal"] }] }
  ]
}
```

---

## Common Query Patterns

### Basic Service Query

Query logs from a specific service:

```shell
curl -X POST 'https://app.last9.io/api/v4/organizations/{org}/logs/api/v2/query_range/json?start=1743500000000000000&end=1743510000000000000&limit=100&direction=backward&region=ap-south-1' \
  -H 'X-LAST9-API-TOKEN: Bearer <ACCESS_TOKEN>' \
  -H 'Content-Type: application/json' \
  -d '{
    "pipeline": [
      {
        "type": "filter",
        "query": {
          "$and": [
            { "$eq": ["service", "api-gateway"] }
          ]
        }
      }
    ]
  }'
```

### Filter by Multiple Conditions

Query logs with multiple conditions (service AND log level):

```shell
curl -X POST 'https://app.last9.io/api/v4/organizations/{org}/logs/api/v2/query_range/json?start=1743500000000000000&end=1743510000000000000&limit=100' \
  -H 'X-LAST9-API-TOKEN: Bearer <ACCESS_TOKEN>' \
  -H 'Content-Type: application/json' \
  -d '{
    "pipeline": [
      {
        "type": "filter",
        "query": {
          "$and": [
            { "$eq": ["service", "payment-service"] },
            { "$eq": ["level", "error"] }
          ]
        }
      }
    ]
  }'
```

### Text Search

Search for logs containing specific text:

```shell
curl -X POST 'https://app.last9.io/api/v4/organizations/{org}/logs/api/v2/query_range/json?start=1743500000000000000&end=1743510000000000000&limit=50' \
  -H 'X-LAST9-API-TOKEN: Bearer <ACCESS_TOKEN>' \
  -H 'Content-Type: application/json' \
  -d '{
    "pipeline": [
      {
        "type": "filter",
        "query": {
          "$and": [
            { "$eq": ["service", "api-gateway"] },
            { "$contains": ["body", "timeout"] }
          ]
        }
      }
    ]
  }'
```

### Regex Pattern Matching

Use regex to match complex patterns:

```shell
curl -X POST 'https://app.last9.io/api/v4/organizations/{org}/logs/api/v2/query_range/json?start=1743500000000000000&end=1743510000000000000&limit=100' \
  -H 'X-LAST9-API-TOKEN: Bearer <ACCESS_TOKEN>' \
  -H 'Content-Type: application/json' \
  -d '{
    "pipeline": [
      {
        "type": "filter",
        "query": {
          "$and": [
            { "$eq": ["service", "api-gateway"] },
            { "$regex": ["body", "error.*timeout|connection.*refused"] }
          ]
        }
      }
    ]
  }'
```

### OR Conditions with Where Stage

Query logs matching any of multiple conditions using a where stage:

```shell
curl -X POST 'https://app.last9.io/api/v4/organizations/{org}/logs/api/v2/query_range/json?start=1743500000000000000&end=1743510000000000000&limit=100' \
  -H 'X-LAST9-API-TOKEN: Bearer <ACCESS_TOKEN>' \
  -H 'Content-Type: application/json' \
  -d '{
    "pipeline": [
      {
        "type": "filter",
        "query": {
          "$and": [
            { "$eq": ["service", "api-gateway"] }
          ]
        }
      },
      {
        "type": "where",
        "query": {
          "$or": [
            { "$eq": ["level", "error"] },
            { "$eq": ["level", "fatal"] }
          ]
        }
      }
    ]
  }'
```

---

## Response Format

### Success Response Structure

A successful response returns a JSON object with a `status` field and a `data` object containing the results:

```json
{
  "status": "success",
  "data": {
    "resultType": "streams",
    "result": [
      {
        "stream": {
          "service": "api-gateway",
          "level": "error",
          "env": "production"
        },
        "values": [
          ["1743505000000000000", "Connection timeout after 30s"],
          ["1743504990000000000", "Request failed: upstream unavailable"],
          ["1743504980000000000", "Error processing request"]
        ]
      }
    ],
    "stats": {
      "summary": {
        "bytesProcessedPerSecond": 1048576,
        "linesProcessedPerSecond": 500,
        "totalBytesProcessed": 2097152,
        "totalLinesProcessed": 1000,
        "execTime": 0.25
      }
    }
  }
}
```

### Empty Results

If no logs are found, the `result` field will be `null`:

```json
{
  "status": "success",
  "data": {
    "resultType": "streams",
    "result": null,
    "stats": {
      "summary": {
        "bytesProcessedPerSecond": 0,
        "linesProcessedPerSecond": 0,
        "totalBytesProcessed": 0,
        "totalLinesProcessed": 0,
        "execTime": 0
      }
    }
  }
}
```

---

## Advanced Usage

### Time Range Conversion

The API uses Unix nanoseconds for timestamps. To convert:

```shell
# Current time in nanoseconds
echo $(($(date +%s) * 1000000000))

# Time from 1 hour ago in nanoseconds
echo $((($(date +%s) - 3600) * 1000000000))

# Convert seconds to nanoseconds
SECONDS=1743500000
NANOSECONDS=$((SECONDS * 1000000000))
```

### Querying From Specific Indices

To query logs from a specific index, add the `index` parameter:

#### For Physical Indices

```shell
curl -X POST 'https://app.last9.io/api/v4/organizations/{org}/logs/api/v2/query_range/json?start=1743500000000000000&end=1743510000000000000&limit=100&index=physical_index:Pt_prod_k8s' \
  -H 'X-LAST9-API-TOKEN: Bearer <ACCESS_TOKEN>' \
  -H 'Content-Type: application/json' \
  -d '{
    "pipeline": [
      {
        "type": "filter",
        "query": {
          "$and": [
            { "$eq": ["service", "api-gateway"] }
          ]
        }
      }
    ]
  }'
```

#### For Rehydration Indices

```shell
curl -X POST 'https://app.last9.io/api/v4/organizations/{org}/logs/api/v2/query_range/json?start=1743500000000000000&end=1743510000000000000&limit=100&index=rehydration_index:Rh_prod_archive' \
  -H 'X-LAST9-API-TOKEN: Bearer <ACCESS_TOKEN>' \
  -H 'Content-Type: application/json' \
  -d '{
    "pipeline": [
      {
        "type": "filter",
        "query": {
          "$and": [
            { "$eq": ["service", "api-gateway"] }
          ]
        }
      }
    ]
  }'
```

The format for the index parameter is:

- `physical_index:<index_name>` for physical indices
- `rehydration_index:<index_name>` for rehydration indices

---

## Error Handling

### Error Response Format

```json
{
  "error": {
    "code": "ERROR_CODE",
    "message": "Human-readable error message"
  }
}
```

### Common Errors

| HTTP Status | Error                            | Description                          |
| ----------- | -------------------------------- | ------------------------------------ |
| 400         | Bad Request                      | Invalid query parameters or pipeline |
| 401         | `Authorization token is expired` | Access token has expired             |
| 403         | Forbidden                        | Insufficient permissions             |
| 404         | Not Found                        | Resource not found                   |
| 500         | Internal Server Error            | Server-side error                    |

### Debugging Common Scenarios

#### No Data Returned

If your query returns no data (`"result": null`), check the following:

1. **Field names**: Verify the field names are correct. Field names are case-sensitive. Use [Discover Available Labels](#discover-available-labels) to find valid fields.
2. **Time range**: Ensure your `start` and `end` timestamps cover a period where logs exist. Remember timestamps are in nanoseconds.
3. **Data retention**: Check if the queried time range is within your organization's data retention period.
4. **Pipeline syntax**: Make sure your JSON pipeline is correctly formatted.

#### Invalid Pipeline Error

If you receive a pipeline parsing error:

1. **JSON syntax**: Ensure your pipeline is valid JSON.
2. **Operator format**: Each operator should be an array with exactly 2 elements: `["field", "value"]`.
3. **Stage type**: Verify the `type` field is one of: `filter`, `where`, `parse`.

#### Authorization Issues

If you receive authentication errors:

1. **Token validity**: Verify your access token is valid and not expired. Tokens expire in 24 hours.
2. **Token scope**: Ensure your access token has the required permissions.
3. **Header format**: The header must be `X-LAST9-API-TOKEN: Bearer <TOKEN>` (not `Authorization`).
4. **Token refresh**: If expired, generate a new token using the refresh token. See [Getting Started with API](/docs/getting-started-with-api/).

---

## API Endpoints Summary

| Method | Endpoint                                | Description               |
| ------ | --------------------------------------- | ------------------------- |
| POST   | `/logs/api/v2/query_range/json`         | Query logs with pipeline  |
| GET    | `/logs/api/v1/labels`                   | Discover available labels |
| GET    | `/logs/api/v1/label/{labelName}/values` | Discover label values     |

---

## Troubleshooting

Please get in touch with us on [Discord](https://discord.com/invite/Q3p2EEucx9) or [Email](mailto:support@last9.io) if you have any questions.