Java | Last9 Documentation

Use OpenTelemetry to instrument your Java application and send telemetry data to Last9. You can run OpenTelemetry Collector or send the telemetry directly from the application. Read the setup guide for more details.

Instrumentation packages

Download the OpenTelemetry Java agent JAR file and place it in the project root directory.

curl -L https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases/latest/download/opentelemetry-javaagent.jar -o opentelemetry-javaagent.jar

The latest opentelemetry-javaagent.jar can be found in the releases section of the opentelemetry-java-instrumentation repository.

The agent supports Java 8 and above. To pin a specific version instead of tracking the latest release, use the versioned download URL.

curl -L https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases/download/v2.30.0/opentelemetry-javaagent.jar -o opentelemetry-javaagent.jar

Environment variables

Set the following environment variables.

Replace $last9_otlp_endpoint and $last9_otlp_auth_header with the OTLP endpoint and auth header from Integrations → OpenTelemetry. The auth header starts with Basic.

export OTEL_EXPORTER_OTLP_ENDPOINT=$last9_otlp_endpoint
export OTEL_SERVICE_NAME="<your_service_name>"
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=$last9_otlp_auth_header"
export OTEL_TRACES_EXPORTER=otlp
export OTEL_METRICS_EXPORTER=otlp
export OTEL_LOGS_EXPORTER=otlp
export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
export OTEL_TRACES_SAMPLER="always_on"
export OTEL_RESOURCE_ATTRIBUTES="deployment.environment=local"
export OTEL_INSTRUMENTATION_LOGBACK_APPENDER_EXPERIMENTAL_CAPTURE_MDC_ATTRIBUTES="*"
export OTEL_LOG_LEVEL=error

Depending on your Cloud environment, enable following:

OTEL_RESOURCE_PROVIDERS_AWS_ENABLED=true
OTEL_RESOURCE_PROVIDERS_GCP_ENABLED=true
OTEL_RESOURCE_PROVIDERS_AZURE_ENABLED=true

System Properties

Alternatively, you can also set system properties for the configuration.

# otel.properties file
otel.exporter.otlp.endpoint: $last9_otlp_endpoint
otel.service.name: <service_name>
otel.exporter.otlp.headers: Authorization=$last9_otlp_auth_header
otel.traces.exporter: otlp
otel.metrics.exporter: otlp
otel.logs.exporter: otlp
otel.exporter.otlp.protocol: http/protobuf
otel.traces.sampler: always_on
otel.resource.attributes: "deployment.environment=staging"
otel.instrumentation.logback-appender.experimental.capture-mdc-attributes: "*"

Depending on your Cloud environment, enable following:

otel.resource.providers.aws.enabled: true
otel.resource.providers.gcp.enabled: true
otel.resource.providers.azure.enabled: true

You can either use a properties file or use the runtime syntax for setting system properties as follows.

java -javaagent:path/to/opentelemetry-javaagent.jar -Dotel.exporter.otlp.endpoint=$last9_otlp_endpoint -Dotel.service.name=<service_name> -Dotel.exporter.otlp.headers="Authorization=$last9_otlp_auth_header" -Dotel.traces.exporter=otlp -Dotel.metrics.exporter=otlp -Dotel.logs.exporter=otlp -Dotel.exporter.otlp.protocol=http/protobuf -Dotel.instrumentation.logback-appender.experimental.capture-mdc-attributes="*" -Dotel.traces.sampler=always_on -Dotel.resource.attributes="deployment.environment=staging" -jar myapp.jar

Caveat

Older Java OpenTelemetry may require replacing the space after Basic in $last9_otlp_auth_header with %20. If you receive 401 error from the Last9 Opentelemetry endpoint, update the OTEL_EXPORTER_OTLP_HEADERS as follows.

Authorization=Basic%20<base64-credentials>

Start the application

You can start your application specifying the -javaagent parameter now:

java -javaagent:path/to/opentelemetry-javaagent.jar -jar myapp.jar

You can specify an environment variable that automatically adds the -javaagent parameter to your JVM:

export JAVA_TOOL_OPTIONS="-javaagent:path/to/opentelemetry-javaagent.jar"
java -jar myapp.jar # or current build command

The application will send telemetry data such as logs, traces, and metrics to Last9.

Capture HTTP request/response bodies

Add HTTP request and response body capture to your traces with the java-otel-body-capture extension. Bodies appear as http.request.body and http.response.body span attributes — no changes to your application code required.

Works with Spring Boot 2.x (javax.servlet) and Spring Boot 3.x (jakarta.servlet), as well as any app running on Tomcat, Jetty, or Dropwizard.

Download the extension JAR

Download last9-otel-body-capture.jar from the latest release.

Add it to your startup command

java -javaagent:opentelemetry-javaagent.jar \
     -Dotel.javaagent.extensions=last9-otel-body-capture.jar \
     -jar your-app.jar

Every HTTP span now carries http.request.body and http.response.body as attributes.

Configuration

All options are system properties. The defaults are production-safe out of the box.

Property	Default	Description
`otel.bodycapture.enabled`	`true`	Master switch
`otel.bodycapture.capture_request_body`	`true`	Capture request bodies
`otel.bodycapture.capture_response_body`	`true`	Capture response bodies
`otel.bodycapture.max_body_size_bytes`	`8192`	Truncates larger bodies with `...[TRUNCATED]`
`otel.bodycapture.capture_on_error_only`	`false`	Only attach bodies on 4xx/5xx
`otel.bodycapture.exclude_paths`	`/health,/ready,/metrics`	Skip these path prefixes
`otel.bodycapture.include_paths`	(all)	Restrict to these path prefixes only
`otel.bodycapture.content_types`	`application/json,application/xml,text/plain`	Only capture these content types

java -javaagent:opentelemetry-javaagent.jar \
     -Dotel.javaagent.extensions=last9-otel-body-capture.jar \
     -Dotel.bodycapture.capture_on_error_only=true \
     -Dotel.bodycapture.include_paths=/api/ \
     -jar your-app.jar

PII/PHI redaction at the collector

Redact sensitive data at the OTel Collector so the pattern is centralized and no app redeployment is needed to update it.

The Last9 ingest endpoint is not a collector you can configure. To apply this processor, run your own OpenTelemetry Collector between your application and Last9. For example, run it as a sidecar on AWS ECS. Point the Java agent’s otel.exporter.otlp.endpoint at the collector, and configure the collector to export to Last9.

Add the processor to the collector’s traces pipeline:

processors:
  transform/redact-pii:
    trace_statements:
      - context: span
        statements:
          # SSN
          - replace_pattern(attributes["http.request.body"], "\\b\\d{3}-\\d{2}-\\d{4}\\b", "***-**-****")
          - replace_pattern(attributes["http.response.body"], "\\b\\d{3}-\\d{2}-\\d{4}\\b", "***-**-****")
          # Email
          - replace_pattern(attributes["http.request.body"], "[a-zA-Z0-9._%+\\-]+@[a-zA-Z0-9.\\-]+\\.[a-zA-Z]{2,}", "****@****.***")
          - replace_pattern(attributes["http.response.body"], "[a-zA-Z0-9._%+\\-]+@[a-zA-Z0-9.\\-]+\\.[a-zA-Z]{2,}", "****@****.***")

service:
  pipelines:
    traces:
      processors: [transform/redact-pii, batch]

For a complete working example, see the HTTP body capture example.

Troubleshooting

Please get in touch with us on Discord or Email if you have any questions.