# Enable EC2 Service Discovery with vmagent

> This tutorial walks through setting up service disocvery for EC2 instances with vmagent.

Source: https://last9.io/docs/how-to-enable-ec2-service-discovery-with-vmagent/

## Service Discovery

In the context of monitoring, Service Discovery refers to automatically detecting devices, services, or systems in a network that need to be monitored. Service discovery is significant in cloud environments that use auto-scaling and EC2 instances. These environments often have instances that change rapidly, making manual tracking infeasible from a monitoring point of view. This document lists steps to enable service discovery of EC2 instances so new instances can be monitored as they are created and decommissioned instances can be removed from monitoring, tackling false alerts.

This document assumes that the EC2 instance service discovery will be set up for vmagent to send metrics to Last9 via Remote Write.

:::note
Follow our guide for [setting up vmagent on an EC2 machine](/docs/how-to-setup-vmagent-on-ubuntu/).
:::

Given that vmagent is successfully running on an EC2 Instance, we need to make provisions for vmagent to discover other EC2 instances, that is, scrape targets based on [ec2_sd_config](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#ec2_sd_config).

## Create `ec2-trustee` IAM role with assume role policy

Go to AWS Console → IAM → Roles → Create Role

- Select Trusted Entity
  ![Select Trusted Entity](../../../../assets/content/docs/tutorials/how-to-enable-ec2-service-discovery-with-vmagent/select-trusted-entity.png)
- Do NOT add any permissions and click next
  ![Add Permissions](../../../../assets/content/docs/tutorials/how-to-enable-ec2-service-discovery-with-vmagent/add-permissions.png)
- Name, Review and Create
  ![Create IAM Role Step 1](../../../../assets/content/docs/tutorials/how-to-enable-ec2-service-discovery-with-vmagent/create-iam-role-step-1.png)
  ![Create IAM Role Step 2](../../../../assets/content/docs/tutorials/how-to-enable-ec2-service-discovery-with-vmagent/create-iam-role-step-2.png)

## Attach `ec2-trustee` IAM role to vmagent EC2 Host

EC2 Instances > Select vmagent Instance > Actions > Instance Settings

- Modify IAM Role
  ![Steps to update IAM role](../../../../assets/content/docs/tutorials/how-to-enable-ec2-service-discovery-with-vmagent/select-vmagent-ec2-instance.png)
- Select `ec2-trustee` IAM role and Update
  ![Modify IAM Role](../../../../assets/content/docs/tutorials/how-to-enable-ec2-service-discovery-with-vmagent/modify-iam-role.png)

## Create `vmagent-sd-role` IAM role

Go to AWS Console → IAM → Roles → Create Role

- Select Trusted Entity > Custom Trust Policy with below trust policy
  ![Custom Trust Policy](../../../../assets/content/docs/tutorials/how-to-enable-ec2-service-discovery-with-vmagent/custom-trust-policy.png)

  ```json
  {
    "Version": "2012-10-17",
    "Statement": [
      {
        "Effect": "Allow",
        "Principal": {
          "AWS": "arn:aws:iam::AWS_ACCOUNT_ID:role/ec2-trustee"
        },
        "Action": "sts:AssumeRole"
      }
    ]
  }
  ```

- Add Permissions

  1.  Create Custom Policy with the below policy

            ```json
            {
                "Statement": [
                    {
                        "Action": "ec2:Describe*",
                        "Effect": "Allow",
                        "Resource": "*"
                    }
                ],
                "Version": "2012-10-17"
            }
            ```

  2.  Select Policy and click next
      ![Select Policy Step 1](../../../../assets/content/docs/tutorials/how-to-enable-ec2-service-discovery-with-vmagent/select-policy-step-1.png)
      ![Select Policy Step 2](../../../../assets/content/docs/tutorials/how-to-enable-ec2-service-discovery-with-vmagent/select-policy-step-2.png)

  3.  Name, Review and Create
      - Add `vmagent-sd-role` as the name of the role, review permissions and trusted entities and create role
        ![Add vmagent role](../../../../assets/content/docs/tutorials/how-to-enable-ec2-service-discovery-with-vmagent/add-vmagent-sd-role.png)

## Use the `vmagent-sd-role` ARN in vmagent configuration

Update the `scrape_configs` stanza in your `vmagent.yaml` with the `ec2_sd_configs`
stanza as follows and restart vmagent.

```yaml
# vmagent.yaml

# Check https://prometheus.io/docs/prometheus/latest/configuration/configuration for more details
scrape_configs:
  - job_name: "node-exporter-sd"
    ec2_sd_configs:
      - region: ap-south-1
        role_arn: "__role_arn_with_ec2_read_access__"
        filters:
          - name: tag:namespace
            values:
              - node-exporter
        port: 9100
```

This will discover new EC2 instances automatically using the Service Discovery
mechanism and their metrics will sent to Last9 from vmagent.