# FastAPI

> Instrument your FastAPI application with OpenTelemetry to send traces, metrics, and logs to Last9

Source: https://last9.io/docs/integrations/frameworks/python/fastapi/

Use OpenTelemetry to instrument your FastAPI application and send telemetry data to Last9. This integration provides automatic instrumentation for HTTP requests, database queries, and other operations in your FastAPI application.

You can either run OpenTelemetry Collector as a separate service or send telemetry directly from your application to Last9.

## Prerequisites

Before setting up FastAPI monitoring, ensure you have:

- Python 3.7 or higher installed
- FastAPI application running
- Last9 account with integration credentials
- `pip` package manager available

1. **Install OpenTelemetry Packages**

   Install the required OpenTelemetry packages for FastAPI instrumentation:

   ```bash
   pip install opentelemetry-api opentelemetry-sdk opentelemetry-exporter-otlp opentelemetry-distro
   ```

   You can also add these packages to your `requirements.txt` file and install them using:

   ```bash
   pip install -r requirements.txt
   ```

2. **Set Environment Variables**

   Configure OpenTelemetry environment variables for your FastAPI application. Replace the placeholder values with your actual Last9 credentials:

   ```bash
   export OTEL_SERVICE_NAME=<service_name>
   export OTEL_EXPORTER_OTLP_ENDPOINT={{ .Logs.WriteURL }}
   export OTEL_EXPORTER_OTLP_HEADERS="Authorization={{ .Logs.AuthValue }}"
   export OTEL_TRACES_EXPORTER=otlp
   export OTEL_EXPORTER_OTLP_PROTOCOL="http/protobuf"
   export OTEL_RESOURCE_ATTRIBUTES="service.name=<service_name>,service.version=1.0.0,deployment.environment=local"
   export OTEL_TRACES_SAMPLER="always_on"
   export OTEL_LOG_LEVEL=error
   ```

   **Environment Variables Explained:**

   - `OTEL_SERVICE_NAME`: Name of your FastAPI service
   - `OTEL_EXPORTER_OTLP_ENDPOINT`: Last9 endpoint URL for telemetry data
   - `OTEL_EXPORTER_OTLP_HEADERS`: Authentication header for Last9
   - `OTEL_RESOURCE_ATTRIBUTES`: Service metadata including name, version, and environment

   :::caution[Prefer gRPC in production]
   The OTLP HTTP exporter uses a long-lived `requests.Session`. Load balancers with short idle timeouts (ALB 60s, CloudFront 60s, nginx 75s) can close idle TCP connections, causing intermittent `RemoteDisconnected` tracebacks on the next export.

   Two ways to avoid this:

   - Switch to the gRPC exporter — `pip install opentelemetry-exporter-otlp-proto-grpc` and set `OTEL_EXPORTER_OTLP_PROTOCOL=grpc`. HTTP/2 keepalive pings prevent idle-socket closes.
   - Upgrade `opentelemetry-*` to 1.27 or later. Newer SDKs retry on `ConnectionError` silently; 1.15.x raises the full traceback.

   The `BatchSpanProcessor` retries failed batches either way, so a single log line usually means no data loss. See [Debug OpenTelemetry pipelines](/guides/opentelemetry/how-to-debug-opentelemetry-pipelines/#python-otlp-http-exporter-remotedisconnected) for the full breakdown.
   :::

3. **Bootstrap Automatic Instrumentation**

   FastAPI supports automatic instrumentation with OpenTelemetry. Run the following command to detect and install the required instrumentation packages for your application:

   ```bash
   opentelemetry-bootstrap -a requirements
   ```

   This command analyzes your `requirements.txt` file and outputs the OpenTelemetry instrumentation packages you need. For example:

   - If you use `requests`, it will suggest `opentelemetry-instrumentation-requests`
   - If you use `sqlalchemy`, it will suggest `opentelemetry-instrumentation-sqlalchemy`
   - If you use `redis`, it will suggest `opentelemetry-instrumentation-redis`

4. **Install Instrumentation Packages**

   Install all the recommended instrumentation packages automatically:

   ```bash
   opentelemetry-bootstrap -a install
   ```

   This command installs all the necessary instrumentation packages based on your application's dependencies.

   > **Recommendation**: Last9 recommends using automatic instrumentation for most applications as it provides comprehensive coverage with minimal configuration.

5. **Run Your FastAPI Application**

   Start your FastAPI application with OpenTelemetry instrumentation:

   ```bash
   opentelemetry-instrument fastapi run app.py
   ```

   Alternatively, if you're using uvicorn directly:

   ```bash
   opentelemetry-instrument uvicorn app:app --host 0.0.0.0 --port 8000
   ```

   This command starts your FastAPI application with automatic OpenTelemetry instrumentation enabled. The instrumentation will:

   - Collect traces for HTTP requests and responses
   - Monitor database queries and external API calls
   - Track performance metrics and errors
   - Send all telemetry data to Last9

## What Gets Instrumented

When you use automatic instrumentation with FastAPI, OpenTelemetry automatically captures:

### HTTP Requests

- Request method, URL, and headers
- Response status codes and timing
- Route parameters and query strings
- Request and response body sizes

### Database Operations

- SQL queries and execution times
- Database connection details
- Query parameters and result counts

### External API Calls

- HTTP client requests (using `requests`, `httpx`, etc.)
- Request/response details and timing
- Error handling and retry attempts

### Background Tasks

- FastAPI background task execution
- Task duration and completion status

## Advanced Configuration

### Custom Service Metadata

Enhance your service metadata by updating the `OTEL_RESOURCE_ATTRIBUTES` environment variable:

```bash
export OTEL_RESOURCE_ATTRIBUTES="service.name=fastapi-api,service.version=2.1.0,deployment.environment=production,service.instance.id=api-server-1,team=backend"
```

### Sampling Configuration

Control trace sampling to manage data volume:

```bash
# Always sample (development)
export OTEL_TRACES_SAMPLER="always_on"

# Sample 10% of traces (production)
export OTEL_TRACES_SAMPLER="traceidratio"
export OTEL_TRACES_SAMPLER_ARG="0.1"
```

### Excluding Noisy Endpoints

Health-check and liveness endpoints polled every few seconds generate hundreds of identical, non-actionable spans per hour. Use `OTEL_PYTHON_EXCLUDED_URLS` to drop them before they reach Last9.

```bash
# Comma-separated regex patterns — matched via re.search() on the full URL
export OTEL_PYTHON_EXCLUDED_URLS="health-check,ping,ready,live,metrics"
```

This works automatically with `opentelemetry-instrument` — no code changes needed.

Patterns use `re.search()` so partial matches apply: `health-check` excludes `/health-check`, `/api/health-check`, `/internal/health-check`, and so on.

For framework-specific overrides (useful in multi-framework codebases):

```bash
export OTEL_PYTHON_FASTAPI_EXCLUDED_URLS="health-check,ping,ready"
```

Framework-specific variants take precedence over the generic `OTEL_PYTHON_EXCLUDED_URLS`.

:::note[Custom sampler for advanced filtering]
If you need more control — filtering by HTTP status code, tenant, or business logic — use a custom sampler. See the [trace-filtering example](https://github.com/last9/l9_otel_examples/tree/main/python/trace-filtering) for a drop-in `ParentBased` sampler that also preserves error spans on excluded paths.
:::

### Custom Span Attributes

Add custom attributes to your spans programmatically:

```python
from opentelemetry import trace

tracer = trace.get_tracer(__name__)

@app.get("/users/{user_id}")
async def get_user(user_id: str):
    with tracer.start_as_current_span("get_user") as span:
        span.set_attribute("user.id", user_id)
        span.set_attribute("operation.type", "user_lookup")
        # Your FastAPI logic here
        return {"user_id": user_id}
```

## Verification

1. **Check Application Startup**

   When you start your FastAPI application, you should see OpenTelemetry initialization messages in the console output.

2. **Generate Test Traffic**

   Make some requests to your FastAPI application to generate telemetry data:

   ```bash
   curl http://localhost:8000/
   curl http://localhost:8000/health
   curl http://localhost:8000/api/users
   ```

3. **Verify Data in Last9**

   Log into your Last9 account and check that traces and metrics are being received in [Grafana](https://app.last9.io/grafana).

   Look for:

   - HTTP request traces with timing information
   - Database query spans
   - Error traces for failed requests
   - Performance metrics

## Troubleshooting

### Common Issues

**Missing Instrumentation Packages**
If some operations aren't being traced, run the bootstrap command again:

```bash
opentelemetry-bootstrap -a requirements
opentelemetry-bootstrap -a install
```

**Environment Variables Not Set**
Verify your environment variables are properly configured:

```bash
env | grep OTEL_
```

**Connection Issues**
Check if your application can reach Last9:

```bash
curl -v -H "Authorization: <your-auth-header>" <your-endpoint>
```

**`RemoteDisconnected` tracebacks from the OTLP HTTP exporter**
Load balancers (ALB, nginx, CloudFront) close idle TCP connections after 60–75 seconds. The OTLP HTTP exporter reuses the same connection, so the next export after an idle period hits a closed socket and logs a full traceback. Two fixes:

- Switch to the gRPC exporter — HTTP/2 keepalive prevents the close:
  ```bash
  pip install opentelemetry-exporter-otlp-proto-grpc
  export OTEL_EXPORTER_OTLP_PROTOCOL=grpc
  ```
- Or upgrade `opentelemetry-*` to 1.27+, which retries `ConnectionError` silently.

The `BatchSpanProcessor` retries the failed batch either way, so this is usually noise rather than data loss. See [Debug OpenTelemetry pipelines](/guides/opentelemetry/how-to-debug-opentelemetry-pipelines/#python-otlp-http-exporter-remotedisconnected) for a deeper walkthrough.

### Performance Considerations

- Use sampling in production to control data volume
- Monitor the overhead of instrumentation on your application
- Consider using the OpenTelemetry Collector as a buffer for high-traffic applications

### Best Practices

- Set meaningful service names that reflect your architecture
- Use consistent environment naming across services
- Include version information in resource attributes
- Add custom spans for critical business operations
- Use structured logging alongside tracing

## Supported Libraries

OpenTelemetry provides automatic instrumentation for many popular Python libraries commonly used with FastAPI:

- **HTTP Clients**: `requests`, `httpx`, `aiohttp`
- **Databases**: `sqlalchemy`, `psycopg2`, `pymongo`, `redis`
- **Message Queues**: `celery`, `pika` (RabbitMQ)
- **Cloud Services**: `boto3` (AWS), `google-cloud`

For a complete list of supported libraries, visit the [OpenTelemetry Python Contrib documentation](https://opentelemetry-python-contrib.readthedocs.io/en/latest/index.html).

## Need Help?

If you encounter any issues or have questions:

- Join our [Discord community](https://discord.com/channels/652153247672729619/652153247672729621) for real-time support
- Contact our support team at [support@last9.io](mailto:support@last9.io)