GitHub Copilot

GitHub Copilot emits structured OpenTelemetry telemetry for agent sessions: LLM calls, tool executions, token usage, and latency. Routing this data to Last9 gives you visibility into how your team uses Copilot — without custom instrumentation.

Copilot follows OpenTelemetry GenAI semantic conventions, so spans and metrics use standardized attribute names compatible with any OTel backend, including Last9.

There are two ways to export Copilot telemetry to Last9:

What gets exported

Traces

Each agent interaction produces a hierarchical span tree:

invoke_agent                          [~15s]
  ├── chat gpt-4o                     [~3s]
  ├── execute_tool readFile           [~50ms]
  ├── execute_tool runCommand         [~2s]
  └── chat gpt-4o                     [~4s]

When an agent invokes a subagent (for example via runSubagent), the subagent’s invoke_agent span is parented under the calling execute_tool span.

Prompt and response content is not captured by default. Enable content capture to opt in — see Configuration reference.

Metrics

Events

Copilot Chat also emits OTel events for session starts, tool completions, edit accept/reject feedback, and user thumbs-up/down — useful for acceptance-rate dashboards alongside traces.

Prerequisites

1. Last9 account — Sign up at app.last9.io
2. GitHub Copilot subscription — Individual, Business, or Enterprise plan
3. OTLP credentials — Get your endpoint and auth header from Integrations → OpenTelemetry

Copilot Chat (VS Code)

Use this path when developers use Copilot inside VS Code — agent mode, inline chat, or Copilot CLI sessions launched from the editor.

Additional prerequisites

• VS Code 1.119+
• Copilot Chat extension with OTel support (update to the latest version)

Setup

1. Get your Last9 OTLP credentials

   Navigate to Integrations → OpenTelemetry in your Last9 dashboard. Copy:

   • OTLP Endpoint (e.g., https://otlp-aps1.last9.io:443)
   • Authorization header (e.g., Basic <base64-token>)

2. Enable OTel export

   OTel activates when any of these is true:

   • github.copilot.chat.otel.enabled is true
   • COPILOT_OTEL_ENABLED=true
   • OTEL_EXPORTER_OTLP_ENDPOINT is set
   • Environment variables (recommended)
   • VS Code settings
   • Enterprise rollout

   Export these variables before launching VS Code (or set them in your shell profile / enterprise launch script):

   export COPILOT_OTEL_ENABLED=true
   export OTEL_EXPORTER_OTLP_ENDPOINT="https://<your-last9-otlp-endpoint>"
   export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Basic <your-last9-auth-token>"
   export OTEL_SERVICE_NAME="copilot-chat"
   export OTEL_RESOURCE_ATTRIBUTES="deployment.environment=production,team=<your-team>"

   Environment variables take precedence over VS Code settings.

3. Restart VS Code and send a test prompt

   Open Copilot Chat in agent mode and send any prompt. Wait up to 60 seconds for the first metrics flush.

4. Verify in Last9

   • Traces — filter by service.name = copilot-chat and look for invoke_agent root spans
   • Metrics — search for copilot_chat_agent_invocation_duration or gen_ai_client_token_usage
   • Distinguish agent types — filter spans by gen_ai.agent.name:
     • GitHub Copilot Chat — foreground agent (agent mode)
     • copilotcli — CLI wrapper span
     • claude — Claude agent sessions proxied through VS Code

Copilot Chat configuration reference

Copilot SDK (custom agents)

Use this path when you build custom agents with the GitHub Copilot SDK and CopilotClient.

Additional prerequisites

1. GitHub token — A user-level token with a Copilot subscription behind it:
   • Fine-grained PAT (github_pat_*) — easiest for local dev
   • GitHub App user access token (ghu_*) — recommended for production
   • Classic PATs (ghp_*) and GitHub App installation tokens are not accepted
2. Copilot SDK — Installed in your agent project (Node.js, Python, Go, or .NET)

Setup

1. Get your Last9 OTLP credentials — same as Copilot Chat above.

2. Install the SDK with telemetry support

   • Node.js / TypeScript
   • Python
   • Go
   • .NET

   npm install @github/copilot-sdk

3. Configure telemetry in your agent

   Pass a TelemetryConfig when constructing the CopilotClient. Set the otlpEndpoint to your Last9 OTLP endpoint and include your authorization header.

   • Node.js / TypeScript
   • Python
   • Go
   • .NET

   import { CopilotClient } from "@github/copilot-sdk";
   
   const client = new CopilotClient({
     telemetry: {
       exporterType: "otlp-http",
       otlpEndpoint: "https://<your-last9-otlp-endpoint>",
       sourceName: "my-copilot-agent",
       captureContent: false,
     },
   });

   export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Basic <your-last9-auth-token>"
   export OTEL_SERVICE_NAME="my-copilot-agent"

4. Invoke your agent and verify data

   Run your agent and trigger at least one interaction. Then in Last9:

   • Traces — filter by service.name = my-copilot-agent and look for invoke_agent root spans
   • Metrics — search for copilot_chat_agent_invocation_duration

SDK TelemetryConfig fields

Org-level Copilot metrics (coming soon)

GitHub also exposes organization-level Copilot metrics — seat utilization, acceptance rates, and language breakdown — via the Copilot metrics API. Last9 is adding ingestion of these metrics through the Last9 GitHub App so platform teams can see org-wide Copilot adoption without per-developer OTLP setup.

This complements per-session traces from Copilot Chat; it does not replace them.

What you can do in Last9

Agent latency breakdown (traces)

Each invoke_agent span contains child chat and execute_tool spans. In Last9 APM, expand a trace to see how much time was spent on LLM calls versus tool execution within a single agent turn.

Token and cost tracking (metrics)

gen_ai.client.token.usage is broken down by gen_ai.request.model. Use it to compare token consumption across models, track usage trends, and alert on spikes.

Tool performance (metrics)

copilot_chat.tool.call.duration and copilot_chat.tool.call.count show which tools are called most and which are slowest.

Error rate monitoring (traces)

Failed LLM calls and tool executions set error.type on their spans. Create a Last9 alert on error rate for invoke_agent spans to catch Copilot API degradation before users report it.

Team tagging

Use OTEL_RESOURCE_ATTRIBUTES to tag sessions by team or environment:

export OTEL_RESOURCE_ATTRIBUTES="team=platform,deployment.environment=production"

All signals from that developer’s VS Code session carry these labels, enabling per-team breakdowns in Last9 dashboards.

Troubleshooting

• No traces in Last9 after a Copilot Chat session

  • Confirm VS Code is 1.119+ and Copilot Chat is updated
  • Confirm github.copilot.chat.otel.enabled is true or COPILOT_OTEL_ENABLED=true
  • Restart VS Code after setting environment variables — VS Code must inherit OTEL_EXPORTER_OTLP_* at launch
  • Verify the endpoint has no trailing path (use https://otlp-aps1.last9.io:443, not .../v1/traces)
  • Send at least one agent-mode prompt and wait up to 60 seconds

• No traces from a Copilot SDK agent

  • Confirm exporterType: "otlp-http" — double-check it is not set to "file"
  • Check that OTEL_EXPORTER_OTLP_HEADERS is exported in the same shell session where the agent runs

• 401 / authentication errors

  • Verify the header format: Authorization=Basic <token> (no extra quotes, no Bearer prefix)
  • Regenerate the token from Integrations → OpenTelemetry if it has expired

• Spans appear but no metrics

  • Metrics may take up to 60 seconds to flush after the first invocation
  • Confirm the Copilot Chat extension or SDK version supports metrics export

• Content is missing from spans

  • By default, prompts and responses are not captured — set github.copilot.chat.otel.captureContent or captureContent: true in SDK config

Please get in touch with us on Discord or Email if you have any questions.