# Telegraf

> How to configure Telegraf to send data to Last9 using the Prometheus Remote Write feature

Source: https://last9.io/docs/integrations/others/telegraf/

[Telegraf](https://www.influxdata.com/time-series-platform/telegraf/) is an open-source agent that collects metrics and data from various sources and sends them to other services, such as InfluxDB, Graphite, and Prometheus. This document describes how Telegraf can integrate with Last9.

Last9 accepts [remote write metrics](https://last9.io/blog/what-is-prometheus-remote-write/) in Prometheus / [OpenTelemetry](https://last9.io/guides/opentelemetry/what-is-opentelemetry/) format. Telegraf supports both push-based and pull-based methods for these formats.

## Pre-requisites

1. You have created a Last9 cluster by following the [getting started guide](/docs/onboard/)
2. Clone [last9-integrations](https://github.com/last9/last9-integrations/tree/master/levitate/remote-write/telegraf) GitHub repository contains the sample code with different approaches discussed in this article

## Push via vmagent (recommended)

![Push via vmagent](../../../../../assets/content/docs/integrations/others/telegraf/3c86a78-image.png)

1. A sample application sends metrics to `telegraf`
2. `telegraf` has an [InfluxDB output plugin](https://github.com/influxdata/telegraf/blob/release-1.14/plugins/outputs/influxdb/README.md), which send metrics to `vmagent`
3. `vmagent` [accepts InfluxDB format](https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#how-to-send-data-from-influxdb-compatible-agents-such-as-telegraf) as input and outputs Prometheus format metrics to Last9

You can configure the `vmagent` URL as follows in the telegraf config file.

```shell
###############################################################################
#                                  OUTPUTS                                    #
###############################################################################
[[outputs.influxdb]]
  urls = ["http://vmagent:8429"]
```

A working sample application can be found [here](https://github.com/last9/last9-integrations/tree/master/levitate/remote-write/telegraf/influxdb-to-vmagent).

**This is the recommended approach for pushing metrics from Telegraf to Last9.**

## Push via [Prometheus Remote Write](https://last9.io/blog/how-to-scale-prometheus-remote-write/) plugin

![Push via Prometheus Remote Write plugin](../../../../../assets/content/docs/integrations/others/telegraf/65fb84b-image.png)

1. A sample application sends metrics to `telegraf`
2. `telegraf` has an [HTTP output plugin](https://github.com/influxdata/telegraf/blob/master/plugins/serializers/prometheusremotewrite/README.md), which supports `prometheusremotewrite` format to convert input metrics to Prometheus protobuf format and push them to [Last9](https://last9.io/)

The changes to the telegraf config are as follows:

```shell
## Data format to output.
  data_format = "prometheusremotewrite"

  ## Additional HTTP headers
  [outputs.http.headers]
     Content-Type = "application/x-protobuf"
     Content-Encoding = "snappy"
     X-Prometheus-Remote-Write-Version = "0.1.0"

```

Here is a [sample application](https://github.com/last9/last9-integrations/tree/master/levitate/remote-write/telegraf/http-prometheusremotewrite) with this setup.

## Pull via Prometheus Client plugin

![Pull via Prometheus Client plugin](../../../../../assets/content/docs/integrations/others/telegraf/a9a9064-image.png)

1. A sample application sends metrics to `telegraf`
2. `telegraf` has a [Prometheus output plugin](https://github.com/influxdata/telegraf/blob/master/plugins/outputs/prometheus_client/README.md), which exposes all input metrics on `/metrics` url
3. These metrics are scraped by `vmagent` which remote writes to Last9

This approach also requires a change in the outputs section in the telegraf config as follows. The `9273` port is used in the `vmagent` [config file](https://github.com/last9/last9-integrations/blob/master/levitate/remote-write/telegraf/prometheus-client/vmagent.yaml) which scrapes it for metrics.

```shell
###############################################################################
#                                  OUTPUTS                                    #
###############################################################################

# Configuration for the Prometheus client to spawn
[[outputs.prometheus_client]]
  ## Address to listen on.
  listen = ":9273"
```

A complete example using this approach can be found [here](https://github.com/last9/last9-integrations/tree/master/levitate/remote-write/telegraf/prometheus-client).

## Comparison

| Approach                         | Pros                                                                                                                                          | Cons                                                                                                                                                                                                                                                               |
| :------------------------------- | :-------------------------------------------------------------------------------------------------------------------------------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Push via vmagent                 | Transparent relay from Telegraf to InfluxDB format to Last9.                                                                                  | None                                                                                                                                                                                                                                                               |
| Push via Prometheus remote write | No need to run any additional scraping agent because of the direct push model.                                                                | Histogram-based metrics might be impacted if Telegraf pushes partial results in one chunk. This has been mentioned in [prometheusremotewrite](https://github.com/influxdata/telegraf/blob/master/plugins/serializers/prometheusremotewrite/README.md) plugin docs. |
| Pull via Client plugin           | Metrics are shipped to Last9 using the [standard scrape model](https://last9.io/blog/best-practices-using-and-writing-prometheus-exporters/). | It has an overhead to run `vmagent` or any other scraping tool additionally.                                                                                                                                                                                       |

## Next steps

Create a read token for your Last9 Cluster and follow our guide to [Configure Grafana](/docs/grafana-config/) to visualize the time series data getting sent to Last9.

---

## Troubleshooting

Please get in touch with us on [Discord](https://discord.com/invite/Q3p2EEucx9) or [Email](mailto:support@last9.io) if you have any questions.