Add Traefik Mesh integration (#17585)

* Initial commit

* Add unit and end to end tests

* added more test and assets to integration

* sync CI

* fix dashboard and E2E tests

* fix metadata types

* added more Traefik config metrics

* Add changelog and readme

* remove additional changelog added

* apply readme suggestions

Co-authored-by: Austin Lai <76412946+alai97@users.noreply.github.com>

* apply suggestions to readme

Co-authored-by: Austin Lai <76412946+alai97@users.noreply.github.com>

* added logo and fixed nits

* removed unneeded PY2 import and made tests more readable

---------

Co-authored-by: Austin Lai <76412946+alai97@users.noreply.github.com>

Author: Kyle-Neale

.codecov.yml

@@ -598,6 +598,10 @@ coverage:
         target: 75
         flags:
         - torchserve
+      Traefik_Mesh:
+        target: 75
+        flags:
+        - traefik_mesh
       Traffic_Server:
         target: 75
         flags:
@@ -1440,6 +1444,11 @@ flags:
     paths:
     - torchserve/datadog_checks/torchserve
     - torchserve/tests
+  traefik_mesh:
+    carryforward: true
+    paths:
+    - traefik_mesh/datadog_checks/traefik_mesh
+    - traefik_mesh/tests
   traffic_server:
     carryforward: true
     paths:

.github/workflows/config/labeler.yml

@@ -469,6 +469,8 @@ integration/tomcat:
 - tomcat/**/*
 integration/torchserve:
 - torchserve/**/*
+integration/traefik_mesh:
+- traefik_mesh/**/*
 integration/traffic_server:
 - traffic_server/**/*
 integration/twemproxy:

.github/workflows/test-all.yml

@@ -3355,6 +3355,25 @@ jobs:
       test-py3: ${{ inputs.test-py3 }}
       minimum-base-package: ${{ inputs.minimum-base-package }}
     secrets: inherit
+  j41a575a:
+    uses: ./.github/workflows/test-target.yml
+    with:
+      job-name: Traefik Mesh
+      target: traefik_mesh
+      platform: linux
+      runner: '["ubuntu-22.04"]'
+      repo: "${{ inputs.repo }}"
+      python-version: "${{ inputs.python-version }}"
+      standard: ${{ inputs.standard }}
+      latest: ${{ inputs.latest }}
+      agent-image: "${{ inputs.agent-image }}"
+      agent-image-py2: "${{ inputs.agent-image-py2 }}"
+      agent-image-windows: "${{ inputs.agent-image-windows }}"
+      agent-image-windows-py2: "${{ inputs.agent-image-windows-py2 }}"
+      test-py2: ${{ inputs.test-py2 }}
+      test-py3: ${{ inputs.test-py3 }}
+      minimum-base-package: ${{ inputs.minimum-base-package }}
+    secrets: inherit
   j8a1b5bb:
     uses: ./.github/workflows/test-target.yml
     with:

traefik_mesh/CHANGELOG.md

@@ -0,0 +1,4 @@
+# CHANGELOG - traefik_mesh
+
+<!-- towncrier release notes start -->
+

traefik_mesh/README.md

@@ -0,0 +1,79 @@
+# Agent Check: traefik_mesh
+
+## Overview
+
+Traefik Mesh is a lightweight and easy-to-deploy service mesh that offers advanced traffic management, security, and observability features for microservices applications, leveraging the capabilities of Traefik Proxy. With Datadog's Traefik integration, you can:
+- Obtain insights into the traffic entering your service mesh.
+- Gain critical insights into the performance, reliability, and security of individual services within your mesh which ensures your services are operating efficiently while also helping to identify and resolve issues quickly.
+- Gain detailed insights into the internal traffic flows within your service mesh which helps monitor performance and ensure reliability.
+
+This check monitors [Traefik Mesh][1] through the Datadog Agent.
+
+## Setup
+
+Follow the instructions below to install and configure this check for an Agent running on a host. For containerized environments, see the [Autodiscovery Integration Templates][3] for guidance on applying these instructions.
+
+### Installation
+
+Starting from Agent release v7.55.0, the Traefik Mesh check is included in the [Datadog Agent][2] package. No additional installation is needed on your server.
+
+**Note**: This check requires Agent v7.55.0 or later.
+
+### Configuration
+
+Traefik Mesh can be configured to expose Prometheus-formatted metrics. The Datadog Agent can collect these metrics using the integration described below. Follow the instructions to configure data collection for your Traefik Mesh instances. For the required configurations to expose the Prometheus metrics, see the [Observability page in the official Traefik Mesh documentation][10].
+
+In addition, a small subset of metrics can be collected by communicating with different API endpoints. Specifically:
+- `/api/version`: Version information on the Traefik proxy.
+- `/api/status/nodes`: Ready status of nodes visible by the Traefik [controller][12].
+- `/api/status/readiness`: Ready status of the Traefik controller.
+
+**Note**: This check uses [OpenMetrics][11] for metric collection, which requires Python 3.
+
+#### Containerized
+##### Metric collection
+
+Make sure that the Prometheus-formatted metrics are exposed in your Traefik Mesh cluster. You can configure and customize this by following the instructions on the [Observability page in the official Traefik Mesh documentation][10]. In order for the Agent to start collecting metrics, the Traefik Mesh pods need to be annotated. For more information about annotations, refer to the [Autodiscovery Integration Templates][3] for guidance. You can find additional configuration options by reviewing the [`traefik_mesh.d/conf.yaml` sample][4].
+
+**Note**: The following metrics can only be collected if they are available. Some metrics are generated only when certain actions are performed.
+
+When configuring the Traefik Mesh check, you can use the following parameters:
+- `openmetrics_endpoint`: This parameter should be set to the location where the Prometheus-formatted metrics are exposed. The default port is `8082`, but it can be configured using the `--entryPoints.metrics.address`. In containerized environments, `%%host%%` can be used for [host autodetection][3].
+- `traefik_proxy_api_endpooint:` This parameter is optional. The default port is `8080` and can be configured using `--entryPoints.traefik.address`. In containerized environments, `%%host%%` can be used for [host autodetection][3].
+- `traefik_controller_api_endpoint`: This parameter is optional. The default port is set to `9000`. 
+
+### Validation
+
+[Run the Agent's status subcommand][6] and look for `traefik_mesh` under the Checks section.
+
+## Data Collected
+
+### Metrics
+
+See [metadata.csv][7] for a list of metrics provided by this integration.
+
+### Events
+
+The Traefik Mesh integration does not include any events.
+
+### Service Checks
+
+See [service_checks.json][8] for a list of service checks provided by this integration.
+
+## Troubleshooting
+
+Need help? Contact [Datadog support][9].
+
+
+[1]: https://traefik.io/
+[2]: https://app.datadoghq.com/account/settings/agent/latest
+[3]: https://docs.datadoghq.com/agent/kubernetes/integrations/
+[4]: https://github.com/DataDog/integrations-core/blob/master/traefik_mesh/datadog_checks/traefik_mesh/data/conf.yaml.example
+[5]: https://docs.datadoghq.com/agent/guide/agent-commands/#start-stop-and-restart-the-agent
+[6]: https://docs.datadoghq.com/agent/guide/agent-commands/#agent-status-and-information
+[7]: https://github.com/DataDog/integrations-core/blob/master/traefik_mesh/metadata.csv
+[8]: https://github.com/DataDog/integrations-core/blob/master/traefik_mesh/assets/service_checks.json
+[9]: https://docs.datadoghq.com/help/
+[10]: https://doc.traefik.io/traefik/observability/metrics/overview/
+[11]: https://docs.datadoghq.com/integrations/openmetrics/
+[12]: https://doc.traefik.io/traefik-mesh/api/

traefik_mesh/assets/configuration/spec.yaml

@@ -0,0 +1,30 @@
+name: Traefik Mesh
+files:
+  - name: traefik_mesh.yaml
+    options:
+    - template: init_config
+      options:
+      - template: init_config/openmetrics
+    - template: instances
+      options:
+      - template: instances/openmetrics
+        overrides:
+          openmetrics_endpoint.required: true
+          openmetrics_endpoint.value.example: http://<PROXY_ENDPOINT>:<METRICS_PORT>
+          openmetrics_endpoint.description: |
+            Endpoint exposing the Traefic Proxy Prometheus metrics. For more information refer to:
+            https://doc.traefik.io/traefik/observability/metrics/prometheus/
+      - name: traefik_proxy_api_endpoint
+        required: false
+        description: URL of the Traefik Mesh proxy to query.
+        value:
+          example: http://<PROXY_ENDPOINT>:<API_PORT>
+          pattern: \w+
+          type: string
+      - name: traefik_controller_api_endpoint
+        required: false
+        description: URL of the Traefik Mesh controller to query.
+        value:
+          example: http://<CONTROLLER_ENDPOINT>:<API_PORT>
+          pattern: \w+
+          type: string