# MongoDB Atlas

> Monitor MongoDB Atlas clusters with metrics, logs, slow query extraction, and memory analysis using OpenTelemetry with Last9

Source: https://last9.io/docs/integrations/databases/mongodb-atlas/

Use OpenTelemetry to instrument your MongoDB Atlas databases and send telemetry data to Last9. This integration collects metrics, logs, events, and extracts slow query details into structured attributes for analysis.

## Prerequisites

Before setting up MongoDB Atlas monitoring, ensure you have:

### MongoDB Atlas API Access

The OpenTelemetry Collector uses the **Atlas Administration API** (not database credentials) to pull metrics and logs. You need to create a programmatic API key pair:

1. In MongoDB Atlas, go to **Project** → **Access Manager** → **API Keys** → **Create API Key**
2. Set a description (e.g., `last9-monitoring`) and select the **Project Data Access Read Only** role

![Create API Key with Project Read Only role](../../../../../../assets/content/docs/integrations/databases/mongoatlas-api-keys.png)

3. Click **Next** to generate the key pair. **Copy both keys immediately** — the private key is shown only once

![Copy the Public Key and Private Key](../../../../../../assets/content/docs/integrations/databases/mongoatlas-keys.png)

4. Optionally add your collector's IP to the **API Access List** (or leave empty to allow all IPs)
5. Click **Done**

**Required Permissions by feature:**

| Feature             | Minimum Role                                                     |
| ------------------- | ---------------------------------------------------------------- |
| Metrics only        | `Project Data Access Read Only` + `Project Observability Viewer` |
| Metrics + host logs | `Project Data Access Read Only` + `Project Observability Viewer` |
| Project events      | `Project Read Only`                                              |
| Organization events | `Organization Member`                                            |
| Access logs         | `Project Owner` or `Organization Owner`                          |
| Audit logs          | `Project Data Access Read Only` + audit logging enabled          |

:::note
These are **Atlas Admin API keys**, not database user credentials. Database users (created under Database Access) authenticate to the database itself. API keys authenticate to the Atlas management API for monitoring.
:::

### Enable Required Features

1. **Database Auditing** (optional): Enable in MongoDB Atlas Project Settings if you want to collect audit logs
2. **Monitoring API**: Ensure monitoring APIs are accessible for your project
3. **Project Configuration**: Note your Project Names and Cluster Names for configuration

1.  **Install OpenTelemetry Collector**

**AMD64**

    ```bash
    sudo apt-get update
    sudo apt-get -y install wget systemctl
    wget https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/v0.128.0/otelcol-contrib_0.128.0_linux_amd64.deb
    sudo dpkg -i otelcol-contrib_0.128.0_linux_amd64.deb
    ```

**ARM64**

    ```bash
    sudo apt-get update
    sudo apt-get -y install wget systemctl
    wget https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/v0.128.0/otelcol-contrib_0.128.0_linux_arm64.deb
    sudo dpkg -i otelcol-contrib_0.128.0_linux_arm64.deb
    ```

2.  **Configure OpenTelemetry Collector**

**Basic Metrics Collection**

    Use the following configuration for collecting MongoDB Atlas metrics:

    ```yaml
    receivers:
      mongodbatlas:
        public_key: "${env:MONGODB_ATLAS_PUBLIC_KEY}"
        private_key: "${env:MONGODB_ATLAS_PRIVATE_KEY}"
        granularity: "PT1M"
        collection_interval: 3m
        projects:
          - name: "YOUR_PROJECT_NAME"
            include_clusters: ["YOUR_CLUSTER_NAME"]
        metrics:
          mongodbatlas.process.cache.ratio:
            enabled: true
        retry_on_failure:
          enabled: true
          initial_interval: 5s
          max_interval: 30s
          max_elapsed_time: 5m

    processors:
      batch:
        timeout: 5s
        send_batch_size: 10000
        send_batch_max_size: 10000
      resourcedetection/system:
        detectors: ["system"]
        system:
          hostname_sources: ["os"]
      resourcedetection/cloud:
        detectors: ["gcp", "ec2", "azure"]

    exporters:
      otlp/last9:
        endpoint: "{{ .Logs.WriteURL }}"
        headers:
          "Authorization": "{{ .Logs.AuthValue }}"
      debug:
        verbosity: detailed

    service:
      pipelines:
        metrics:
          receivers: [mongodbatlas]
          processors: [batch, resourcedetection/system, resourcedetection/cloud]
          exporters: [otlp/last9]
    ```

**Metrics + Logs Collection**

    To collect both metrics and logs from MongoDB Atlas. Host logs include slow query entries which are extracted into structured attributes for analysis.

    ```yaml
    receivers:
      mongodbatlas:
        public_key: "${env:MONGODB_ATLAS_PUBLIC_KEY}"
        private_key: "${env:MONGODB_ATLAS_PRIVATE_KEY}"
        granularity: "PT1M"
        collection_interval: 3m
        projects:
          - name: "YOUR_PROJECT_NAME"
            include_clusters: ["YOUR_CLUSTER_NAME"]
        metrics:
          mongodbatlas.process.cache.ratio:
            enabled: true
        logs:
          enabled: true
          projects:
            - name: "YOUR_PROJECT_NAME"
              collect_host_logs: true
              collect_audit_logs: false
              include_clusters: ["YOUR_CLUSTER_NAME"]
        retry_on_failure:
          enabled: true
          initial_interval: 5s
          max_interval: 30s
          max_elapsed_time: 5m

    processors:
      batch:
        timeout: 5s
        send_batch_size: 10000
        send_batch_max_size: 10000
      resourcedetection/system:
        detectors: ["system"]
        system:
          hostname_sources: ["os"]
      resourcedetection/cloud:
        detectors: ["gcp", "ec2", "azure"]
      transform/add_timestamp:
        log_statements:
          - context: log
            statements:
              - set(observed_time, Now())
              - set(time_unix_nano, observed_time_unix_nano) where time_unix_nano == 0
      transform/add_service:
        log_statements:
          - context: resource
            statements:
              - set(attributes["service.name"], "mongodb-atlas")
              - set(attributes["deployment.environment"], "<env>")
      transform/slow_queries:
        log_statements:
          - context: log
            conditions:
              - IsMatch(body, "Slow query")
            statements:
              - merge_maps(attributes, ParseJSON(body), "upsert") where IsMatch(body, "\"msg\":\"Slow query\"")
              - set(attributes["db.namespace"], attributes["attr"]["ns"]) where attributes["attr"]["ns"] != nil
              - set(attributes["db.operation.duration_ms"], attributes["attr"]["durationMillis"]) where attributes["attr"]["durationMillis"] != nil
              - set(attributes["db.plan_summary"], attributes["attr"]["planSummary"]) where attributes["attr"]["planSummary"] != nil
              - set(attributes["db.keys_examined"], attributes["attr"]["keysExamined"]) where attributes["attr"]["keysExamined"] != nil
              - set(attributes["db.docs_examined"], attributes["attr"]["docsExamined"]) where attributes["attr"]["docsExamined"] != nil
              - set(attributes["db.rows_affected"], attributes["attr"]["nreturned"]) where attributes["attr"]["nreturned"] != nil
              - set(attributes["db.query_hash"], attributes["attr"]["queryHash"]) where attributes["attr"]["queryHash"] != nil
              - set(attributes["db.system"], "mongodb")
              - set(attributes["slow_query"], true)
              - set(severity_text, "WARN") where attributes["attr"]["durationMillis"] != nil and attributes["attr"]["durationMillis"] > 100

    exporters:
      otlp/last9:
        endpoint: "{{ .Logs.WriteURL }}"
        headers:
          "Authorization": "{{ .Logs.AuthValue }}"
      debug:
        verbosity: detailed

    service:
      pipelines:
        logs:
          receivers: [mongodbatlas]
          processors:
            [
              batch,
              resourcedetection/system,
              resourcedetection/cloud,
              transform/add_timestamp,
              transform/add_service,
              transform/slow_queries,
            ]
          exporters: [otlp/last9]
        metrics:
          receivers: [mongodbatlas]
          processors: [batch, resourcedetection/system, resourcedetection/cloud]
          exporters: [otlp/last9]
    ```

**Complete Configuration with Events and Access Logs**

    For comprehensive monitoring, including events, access logs, and slow query extraction:

    ```yaml
    extensions:
      file_storage:
        directory: ./storage
        create_directory: true

    receivers:
      mongodbatlas:
        public_key: "${env:MONGODB_ATLAS_PUBLIC_KEY}"
        private_key: "${env:MONGODB_ATLAS_PRIVATE_KEY}"
        granularity: "PT1M"
        collection_interval: 3m
        storage: file_storage
        projects:
          - name: "YOUR_PROJECT_NAME"
            include_clusters: ["YOUR_CLUSTER_NAME"]
        metrics:
          mongodbatlas.process.cache.ratio:
            enabled: true
        logs:
          enabled: true
          projects:
            - name: "YOUR_PROJECT_NAME"
              collect_host_logs: true
              collect_audit_logs: true
              include_clusters: ["YOUR_CLUSTER_NAME"]
              access_logs:
                enabled: true
                page_size: 20000
                max_pages: 10
                poll_interval: 5m
        events:
          projects:
            - name: "YOUR_PROJECT_NAME"
          organizations:
            - id: "YOUR_ORGANIZATION_ID"
          poll_interval: 1m
          page_size: 100
          max_pages: 25
        alerts:
          enabled: true
          mode: poll
          projects:
            - name: "YOUR_PROJECT_NAME"
              include_clusters: ["YOUR_CLUSTER_NAME"]
          poll_interval: 5m
          page_size: 100
          max_pages: 10
        retry_on_failure:
          enabled: true
          initial_interval: 5s
          max_interval: 30s
          max_elapsed_time: 5m

    processors:
      batch:
        timeout: 5s
        send_batch_size: 10000
        send_batch_max_size: 10000
      resourcedetection/system:
        detectors: ["system"]
        system:
          hostname_sources: ["os"]
      resourcedetection/cloud:
        detectors: ["gcp", "ec2", "azure"]
      transform/add_timestamp:
        log_statements:
          - context: log
            statements:
              - set(observed_time, Now())
              - set(time_unix_nano, observed_time_unix_nano) where time_unix_nano == 0
      transform/add_service:
        log_statements:
          - context: resource
            statements:
              - set(attributes["service.name"], "mongodb-atlas")
              - set(attributes["deployment.environment"], "<env>")
      transform/slow_queries:
        log_statements:
          - context: log
            conditions:
              - IsMatch(body, "Slow query")
            statements:
              - merge_maps(attributes, ParseJSON(body), "upsert") where IsMatch(body, "\"msg\":\"Slow query\"")
              - set(attributes["db.namespace"], attributes["attr"]["ns"]) where attributes["attr"]["ns"] != nil
              - set(attributes["db.operation.duration_ms"], attributes["attr"]["durationMillis"]) where attributes["attr"]["durationMillis"] != nil
              - set(attributes["db.plan_summary"], attributes["attr"]["planSummary"]) where attributes["attr"]["planSummary"] != nil
              - set(attributes["db.keys_examined"], attributes["attr"]["keysExamined"]) where attributes["attr"]["keysExamined"] != nil
              - set(attributes["db.docs_examined"], attributes["attr"]["docsExamined"]) where attributes["attr"]["docsExamined"] != nil
              - set(attributes["db.rows_affected"], attributes["attr"]["nreturned"]) where attributes["attr"]["nreturned"] != nil
              - set(attributes["db.query_hash"], attributes["attr"]["queryHash"]) where attributes["attr"]["queryHash"] != nil
              - set(attributes["db.system"], "mongodb")
              - set(attributes["slow_query"], true)
              - set(severity_text, "WARN") where attributes["attr"]["durationMillis"] != nil and attributes["attr"]["durationMillis"] > 100

    exporters:
      otlp/last9:
        endpoint: "{{ .Logs.WriteURL }}"
        headers:
          "Authorization": "{{ .Logs.AuthValue }}"
      debug:
        verbosity: detailed

    service:
      extensions: [file_storage]
      pipelines:
        logs:
          receivers: [mongodbatlas]
          processors:
            [
              batch,
              resourcedetection/system,
              resourcedetection/cloud,
              transform/add_timestamp,
              transform/add_service,
              transform/slow_queries,
            ]
          exporters: [otlp/last9]
        metrics:
          receivers: [mongodbatlas]
          processors: [batch, resourcedetection/system, resourcedetection/cloud]
          exporters: [otlp/last9]
    ```

3.  **Environment Variables**

    Set the following environment variables before starting the collector:

    ```bash
    export MONGODB_ATLAS_PUBLIC_KEY="your_public_key"
    export MONGODB_ATLAS_PRIVATE_KEY="your_private_key"
    ```

4.  **Start Collector**

    ```bash
    otelcol-contrib --config /etc/otelcol-contrib/config.yaml
    ```

## Memory Metrics

The Atlas receiver collects memory metrics at both the MongoDB process and system level. These OTLP metrics are converted to Prometheus format in Last9:

| OTLP Metric                                | Last9 Metric                                         | Description                                                       | Labels                                                                      |
| ------------------------------------------ | ---------------------------------------------------- | ----------------------------------------------------------------- | --------------------------------------------------------------------------- |
| `mongodbatlas.process.memory.usage`        | `mongodbatlas_process_memory_usage_bytes`            | MongoDB process memory                                            | `memory_state`: `resident`, `virtual`, `mapped`, `computed`                 |
| `mongodbatlas.system.memory.usage.average` | `mongodbatlas_system_memory_usage_average_kibibytes` | System memory average                                             | `memory_status`: `used`, `free`, `available`, `buffers`, `cached`, `shared` |
| `mongodbatlas.system.memory.usage.max`     | `mongodbatlas_system_memory_usage_max_kibibytes`     | Peak system memory                                                | `memory_status`: same as above                                              |
| `mongodbatlas.process.cache.size`          | `mongodbatlas_process_cache_size_bytes`              | WiredTiger cache                                                  | `cache_status`: `dirty`, `used`                                             |
| `mongodbatlas.process.cache.io`            | `mongodbatlas_process_cache_io_bytes`                | WiredTiger cache throughput                                       | `cache_direction`: `read_into`, `written_from`                              |
| `mongodbatlas.process.cache.ratio`         | `mongodbatlas_process_cache_ratio_percent`           | WiredTiger cache fill and dirty ratios (requires explicit enable) | `cache_ratio_type`: `cache_fill`, `dirty_fill`                              |

**Calculating memory usage percentage in Last9:**

```promql
# MongoDB process memory as % of total system memory
mongodbatlas_process_memory_usage_bytes{memory_state="resident"}
/ on()
(mongodbatlas_system_memory_usage_average_kibibytes{memory_status="used"} + mongodbatlas_system_memory_usage_average_kibibytes{memory_status="free"})
/ 1024 * 100
```

:::note
The process memory metric uses bytes while system memory uses kibibytes (KiB). The formula above divides by 1024 to convert bytes to KiB before calculating the percentage. WiredTiger cache utilization is available directly via `mongodbatlas_process_cache_ratio_percent` when enabled in the receiver config.
:::

## Slow Query Monitoring

MongoDB Atlas host logs contain slow query entries in structured JSON format (MongoDB 4.4+). The `transform/slow_queries` processor extracts these fields into structured log attributes:

| Attribute                  | Source Field          | Description                                  |
| -------------------------- | --------------------- | -------------------------------------------- |
| `db.namespace`             | `attr.ns`             | Database and collection (e.g., `mydb.users`) |
| `db.operation.duration_ms` | `attr.durationMillis` | Total execution time in milliseconds         |
| `db.plan_summary`          | `attr.planSummary`    | Execution plan (`COLLSCAN`, `IXSCAN`, etc.)  |
| `db.keys_examined`         | `attr.keysExamined`   | Number of index keys scanned                 |
| `db.docs_examined`         | `attr.docsExamined`   | Number of documents scanned                  |
| `db.rows_affected`         | `attr.nreturned`      | Number of documents returned                 |
| `db.query_hash`            | `attr.queryHash`      | Hash identifying the query shape             |
| `db.system`                | —                     | Always set to `mongodb`                      |
| `slow_query`               | —                     | Always set to `true` for slow query logs     |

Slow queries (> 100ms) are automatically elevated to `WARN` severity. To adjust the slow query threshold in Atlas, go to **Project Settings → Advanced** and set the **Slow Operation Threshold** (default: 100ms).

## Configuration Parameters

### Required Parameters

- `public_key`: MongoDB Atlas API public key
- `private_key`: MongoDB Atlas API private key

### Optional Parameters

- `base_url`: (default: `https://cloud.mongodb.com/`) MongoDB Atlas base URL
- `granularity`: (default: `PT1M`) Metrics granularity (see MongoDB Atlas documentation)
- `collection_interval`: (default: `3m`) How often to collect metrics
- `storage`: Component ID of storage extension for preventing data duplication

### Projects Configuration

- `name`: Name of the MongoDB Atlas project
- `include_clusters`: List of specific clusters to monitor (optional)
- `exclude_clusters`: List of clusters to exclude from monitoring (optional)

### Logs Configuration

- `enabled`: Enable log collection
- `collect_host_logs`: (default: `true`) Collect MongoDB host logs
- `collect_audit_logs`: (default: `false`) Collect audit logs (requires enabled audit logging)

### Events Configuration

- `projects`: List of projects to collect events from
- `organizations`: List of organization IDs to collect events from
- `poll_interval`: (default: `1m`) How often to poll for events
- `page_size`: (default: `100`) Number of events per API request
- `max_pages`: (default: `25`) Maximum pages to request per poll

### Access Logs Configuration

- `enabled`: Enable access log collection
- `auth_result`: Filter by authentication result (true/false/unspecified)
- `page_size`: (default: `20000`) Number of access logs per API request
- `max_pages`: (default: `10`) Maximum pages to request per poll
- `poll_interval`: (default: `5m`) How often to poll for access logs

## Important Notes

1. **Collector Version**: Pin the collector to **v0.128.0**. Versions 0.129.0 through 0.144.0 have a regression where the metrics scraper silently fails to start — logs work fine, but no metrics are collected. This is due to an internal API change (`scraperhelper.AddScraper` → `scraperhelper.AddMetricsScraper`).

2. **API Key Roles**: For metrics collection, your API key needs **both** `Project Data Access Read Only` (for log downloads) **and** `Project Observability Viewer` (for metrics). Without the Observability Viewer role, metrics collection silently produces nothing.

3. **Atlas Tier**: The free tier (M0) does not support host log download or full monitoring APIs. Use **M10+** clusters for complete metrics and log collection.

4. **API Rate Limits**: MongoDB Atlas has API rate limits. The recommended polling interval is 5 minutes for logs and metrics.

5. **Storage Extension**: Use a storage extension to prevent data duplication after collector restarts, especially for events, alerts, and access logs.

6. **Project and Cluster Names**: Replace `YOUR_PROJECT_NAME`, `YOUR_CLUSTER_NAME`, and `YOUR_ORGANIZATION_ID` with your actual MongoDB Atlas project names, cluster names, and organization ID.

7. **Audit Logs**: Audit logging must be explicitly enabled in your MongoDB Atlas project settings to collect audit logs.

8. **Access Logs**: Access log collection requires Project Owner or Organization Owner permissions.

## Verification

To verify the setup is working:

1. Check that the collector service is running
2. Review the collector logs for any errors
3. Log into your Last9 account to confirm MongoDB Atlas metrics are being received

## Need Help?

If you encounter any issues or have questions:

- Join our [Discord community](https://discord.com/channels/652153247672729619/652153247672729621) for real-time support
- Contact our support team at [support@last9.io](mailto:support@last9.io)