-
Notifications
You must be signed in to change notification settings - Fork 1.5k
Add the Teleport Integration #16877
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Add the Teleport Integration #16877
Changes from 59 commits
Commits
Show all changes
60 commits
Select commit
Hold shift + click to select a range
f065279
initial integration scaffolding
NouemanKHAL 39c0783
add initial setup for integration tests using docker-compose
NouemanKHAL 3fb1a7a
add first integration test
NouemanKHAL 0a05403
critical test pass
NouemanKHAL 9b6992c
add test_connect_ok
NouemanKHAL 57fbd9e
fix test_connect_exception and passing test_connect_ok
NouemanKHAL e08d530
add test to check common metrics collection
NouemanKHAL 00c9b3d
add metadata.csv
NouemanKHAL 77b9fc0
ddev test teleport -fs
NouemanKHAL afc8f61
delete some comments
NouemanKHAL 688221b
wip: use openmetrics base class
NouemanKHAL d98f972
catch main super.check exception, passing -> first two tests
NouemanKHAL 52e2c73
move exception to test to unit tests
NouemanKHAL a69ac09
add unit tests, setup mocks, fix integration tests
NouemanKHAL 07a4107
assert common metrics in unit tests
NouemanKHAL d67d762
add metrics fixture
NouemanKHAL 0b2be47
add teleport_cache_stale_events mocks
NouemanKHAL f6ac1d3
reporting common metrics
NouemanKHAL db3a851
rename 'version' tag to 'teleport_version'
NouemanKHAL cc16d1c
format
NouemanKHAL 5818006
remove docker-compose down nothing behavior
NouemanKHAL 04f73f6
update metadata.csv metric names format
NouemanKHAL deff960
update unit and integration tests to match the new metric names
NouemanKHAL 95e88f0
fix implementation to pass the tests
NouemanKHAL 6a672d5
[PLINT-302] Report Teleport Proxy metrics (#17018)
NouemanKHAL 682b734
Merge remote-tracking branch 'origin/master' into noueman/add-telepor…
NouemanKHAL 5dce18b
[PLINT-303] Report Teleport Auth Service and Backends metrics (#17050)
NouemanKHAL 0cd765b
Refactor and cleanup Teleport Integration (#17084)
NouemanKHAL b241f86
[PLINT-304] Report metrics for the Teleport SSH Service (#17111)
NouemanKHAL 322cc3f
[PLINT-306] Report metrics for the Teleport Kubernetes Service (#17113)
NouemanKHAL 4874206
[PLINT-305] Report metrics for the Teleport Database Service (#17114)
NouemanKHAL b12f270
[PLINT-308] Report metrics for the Teleport Enhanced Session Recordin…
NouemanKHAL e41aece
[PLINT-307] Report metrics for the Teleport internal Prometheus (#17117)
NouemanKHAL 6d02272
[PLINT-326] Add `teleport_service` tag to Teleport metrics (#17216)
NouemanKHAL f39c299
[PLINT-331] Update `metadata.csv` for the Teleport Integration (#17241)
NouemanKHAL 338eed4
[PLINT-325] Update Teleport Configuration spec (#17261)
NouemanKHAL 9797165
Merge branch 'master' into noueman/add-teleport-integration
NouemanKHAL 122665d
sort metadata.csv
NouemanKHAL f479d4f
ddev validate ci -s
NouemanKHAL 44e3feb
Merge branch 'master' into noueman/add-teleport-integration
NouemanKHAL dd59bf3
add DEFAULT_METRIC_LIMIT to fix the openmetrics validation-n
NouemanKHAL 889d463
update classifier_tags
NouemanKHAL 33124fa
add description to the manifest
NouemanKHAL 175e732
fix tile description
NouemanKHAL 8d81e36
update manifest check for metrics
NouemanKHAL dedfe7b
[WIP] add dashboard
NouemanKHAL ad2f8b3
add dashboard entry in the manifest.json
NouemanKHAL d40b947
delele monitors, logs, and saved_views entries from the manifest.json
NouemanKHAL 008913f
remove extra comma in manifest.json
NouemanKHAL f1a1135
changelog
NouemanKHAL aaf10dc
Merge branch 'master' into noueman/add-teleport-integration
NouemanKHAL 981445b
ddev validate label --sync
NouemanKHAL 9bfdc06
[PLINT-362] Standardize metric names (#17383)
NouemanKHAL 198a720
Send Teleport service check as a metric (#17441)
NouemanKHAL fb11765
Update metadata.csv with descriptions and units (#17456)
NouemanKHAL 229dce9
Merge branch 'master' into noueman/add-teleport-integration
NouemanKHAL 8c3dfda
fix manifest.json errors
NouemanKHAL e56a2a4
update README.md with prerequisites section
NouemanKHAL 1c9fc65
delete dashboard
NouemanKHAL 892dafc
Apply suggestions from code review
NouemanKHAL File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
# CHANGELOG - Teleport | ||
|
||
## 1.0.0 / 2024-02-07 | ||
|
||
***Added***: | ||
|
||
* Initial Release | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,63 @@ | ||
# Agent Check: Teleport | ||
|
||
## Overview | ||
|
||
This check monitors [Teleport][1] through the Datadog Agent. | ||
|
||
## Setup | ||
|
||
Follow the instructions below to install and configure this check for an Agent running on a host. For containerized environments, see the [Autodiscovery Integration Templates][3] for guidance on applying these instructions. | ||
|
||
### Installation | ||
|
||
The Teleport check is included in the [Datadog Agent][2] package. | ||
No additional installation is needed on your server. | ||
|
||
### Prerequisites | ||
|
||
The Teleport check gathers Teleport's metrics and performance data using two distinct endpoints: | ||
- The [Health endpoint](https://goteleport.com/docs/management/diagnostics/monitoring/#healthz) provides the overall health status of your Teleport instance. | ||
- The [OpenMetrics endpoint](https://goteleport.com/docs/reference/metrics/#auth-service-and-backends) extracts metrics on the Teleport instance and the various services operating within that instance. | ||
|
||
These endpoints aren't activated by default. To enable the diagnostic HTTP endpoints in your Teleport instance, please refer to the public Teleport [documentation](https://goteleport.com/docs/management/diagnostics/monitoring/#enable-health-monitoring). | ||
|
||
### Configuration | ||
|
||
1. Edit the `teleport.d/conf.yaml` file, in the `conf.d/` folder at the root of your Agent's configuration directory to start collecting your teleport performance data. See the [sample teleport.d/conf.yaml][4] for all available configuration options. | ||
|
||
2. [Restart the Agent][5]. | ||
|
||
### Validation | ||
|
||
[Run the Agent's status subcommand][6] and look for `teleport` under the Checks section. | ||
|
||
## Data Collected | ||
|
||
### Metrics | ||
|
||
See [metadata.csv][7] for a list of metrics provided by this integration. | ||
|
||
### Events | ||
|
||
The Teleport integration does not include any events. | ||
|
||
### Service Checks | ||
|
||
The Teleport integration does not include any service checks. | ||
|
||
See [service_checks.json][8] for a list of service checks provided by this integration. | ||
|
||
## Troubleshooting | ||
|
||
Need help? Contact [Datadog support][9]. | ||
|
||
|
||
[1]: **LINK_TO_INTEGRATION_SITE** | ||
[2]: https://app.datadoghq.com/account/settings/agent/latest | ||
[3]: https://docs.datadoghq.com/agent/kubernetes/integrations/ | ||
[4]: https://github.com/DataDog/integrations-core/blob/master/teleport/datadog_checks/teleport/data/conf.yaml.example | ||
[5]: https://docs.datadoghq.com/agent/guide/agent-commands/#start-stop-and-restart-the-agent | ||
[6]: https://docs.datadoghq.com/agent/guide/agent-commands/#agent-status-and-information | ||
[7]: https://github.com/DataDog/integrations-core/blob/master/teleport/metadata.csv | ||
[8]: https://github.com/DataDog/integrations-core/blob/master/teleport/assets/service_checks.json | ||
[9]: https://docs.datadoghq.com/help/ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,23 @@ | ||
name: Teleport | ||
files: | ||
- name: teleport.yaml | ||
options: | ||
- template: init_config | ||
options: | ||
- template: init_config/default | ||
- template: instances | ||
options: | ||
- name: "teleport_url" | ||
required: true | ||
description: "The Teleport URL to connect to." | ||
value: | ||
type: string | ||
example: "http://127.0.0.1" | ||
- name: "diag_port" | ||
description: "The Teleport Diagnostic Port." | ||
value: | ||
type: integer | ||
example: 3000 | ||
|
||
|
||
- template: instances/default |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
[] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
Add the Teleport Integration |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
# (C) Datadog, Inc. 2024-present | ||
# All rights reserved | ||
# Licensed under a 3-clause BSD style license (see LICENSE) | ||
__path__ = __import__('pkgutil').extend_path(__path__, __name__) # type: ignore |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
# (C) Datadog, Inc. 2024-present | ||
# All rights reserved | ||
# Licensed under a 3-clause BSD style license (see LICENSE) | ||
__version__ = '1.0.0' | ||
NouemanKHAL marked this conversation as resolved.
Show resolved
Hide resolved
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
# (C) Datadog, Inc. 2024-present | ||
# All rights reserved | ||
# Licensed under a 3-clause BSD style license (see LICENSE) | ||
from .__about__ import __version__ | ||
from .check import TeleportCheck | ||
|
||
__all__ = ['__version__', 'TeleportCheck'] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,88 @@ | ||
# (C) Datadog, Inc. 2024-present | ||
# All rights reserved | ||
# Licensed under a 3-clause BSD style license (see LICENSE) | ||
|
||
from datadog_checks.base import OpenMetricsBaseCheckV2 | ||
from datadog_checks.base.checks.openmetrics.v2.transform import get_native_dynamic_transformer | ||
|
||
from .metrics import METRIC_MAP, METRIC_MAP_BY_SERVICE | ||
|
||
|
||
class TeleportCheck(OpenMetricsBaseCheckV2): | ||
__NAMESPACE__ = 'teleport' | ||
DEFAULT_METRIC_LIMIT = 0 | ||
DEFAULT_DIAG_PORT = 3000 | ||
|
||
def __init__(self, name, init_config, instances): | ||
super().__init__(name, init_config, instances) | ||
self.check_initializations.appendleft(self._parse_config) | ||
self.check_initializations.append(self._configure_additional_transformers) | ||
|
||
def check(self, _): | ||
try: | ||
health_endpoint = f"{self.diag_addr}/healthz" | ||
response = self.http.get(health_endpoint) | ||
response.raise_for_status() | ||
self.count("health.up", 1, tags=["teleport_status:ok"]) | ||
except Exception as e: | ||
self.log.error( | ||
"Cannot connect to Teleport HTTP diagnostic health endpoint '%s': %s.\nPlease make sure to enable Teleport's diagnostic HTTP endpoints.", # noqa: E501 | ||
health_endpoint, | ||
str(e), | ||
) # noqa: E501 | ||
self.count("health.up", 0, tags=["teleport_status:unreachable"]) | ||
raise | ||
|
||
super().check(_) | ||
|
||
def _parse_config(self): | ||
self.teleport_url = self.instance.get("teleport_url") | ||
self.diag_port = self.instance.get("diag_port", self.DEFAULT_DIAG_PORT) | ||
if self.teleport_url: | ||
self.diag_addr = "{}:{}".format(self.teleport_url, self.diag_port) | ||
self.instance.setdefault("openmetrics_endpoint", "{}/metrics".format(self.diag_addr)) | ||
self.instance.setdefault("rename_labels", {'version': "teleport_version"}) | ||
|
||
def _configure_additional_transformers(self): | ||
metric_transformer = self.scrapers[self.instance['openmetrics_endpoint']].metric_transformer | ||
metric_transformer.add_custom_transformer(r'.*', self.configure_transformer_teleport_metrics(), pattern=True) | ||
|
||
def configure_transformer_teleport_metrics(self): | ||
def transform(_metric, sample_data, _runtime_data): | ||
for sample, tags, hostname in sample_data: | ||
metric_name = _metric.name | ||
metric_type = _metric.type | ||
|
||
# ignore metrics we don't collect | ||
if metric_name not in METRIC_MAP: | ||
continue | ||
|
||
# extract `teleport_service` tag | ||
service = METRIC_MAP_BY_SERVICE.get(metric_name, "teleport") | ||
tags = tags + [f"teleport_service:{service}"] | ||
|
||
# get mapped metric name | ||
new_metric_name = METRIC_MAP[metric_name] | ||
if isinstance(new_metric_name, dict) and "name" in new_metric_name: | ||
new_metric_name = new_metric_name["name"] | ||
|
||
# send metric | ||
metric_transformer = self.scrapers[self.instance['openmetrics_endpoint']].metric_transformer | ||
|
||
if metric_type == "counter": | ||
self.count(new_metric_name + ".count", sample.value, tags=tags, hostname=hostname) | ||
elif metric_type == "gauge": | ||
self.gauge(new_metric_name, sample.value, tags=tags, hostname=hostname) | ||
else: | ||
native_transformer = get_native_dynamic_transformer( | ||
self, new_metric_name, None, metric_transformer.global_options | ||
) | ||
|
||
def add_tag_to_sample(sample, service): | ||
[sample, tags, hostname] = sample | ||
return [sample, tags + [f"teleport_service:{service}"], hostname] | ||
|
||
modified_sample_data = (add_tag_to_sample(x, service) for x in sample_data) | ||
native_transformer(_metric, modified_sample_data, _runtime_data) | ||
|
||
return transform |
24 changes: 24 additions & 0 deletions
24
teleport/datadog_checks/teleport/config_models/__init__.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
# (C) Datadog, Inc. 2024-present | ||
# All rights reserved | ||
# Licensed under a 3-clause BSD style license (see LICENSE) | ||
|
||
# This file is autogenerated. | ||
# To change this file you should edit assets/configuration/spec.yaml and then run the following commands: | ||
# ddev -x validate config -s <INTEGRATION_NAME> | ||
# ddev -x validate models -s <INTEGRATION_NAME> | ||
|
||
from .instance import InstanceConfig | ||
from .shared import SharedConfig | ||
|
||
|
||
class ConfigMixin: | ||
_config_model_instance: InstanceConfig | ||
_config_model_shared: SharedConfig | ||
|
||
@property | ||
def config(self) -> InstanceConfig: | ||
return self._config_model_instance | ||
|
||
@property | ||
def shared_config(self) -> SharedConfig: | ||
return self._config_model_shared |
24 changes: 24 additions & 0 deletions
24
teleport/datadog_checks/teleport/config_models/defaults.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
# (C) Datadog, Inc. 2024-present | ||
# All rights reserved | ||
# Licensed under a 3-clause BSD style license (see LICENSE) | ||
|
||
# This file is autogenerated. | ||
# To change this file you should edit assets/configuration/spec.yaml and then run the following commands: | ||
# ddev -x validate config -s <INTEGRATION_NAME> | ||
# ddev -x validate models -s <INTEGRATION_NAME> | ||
|
||
|
||
def instance_diag_port(): | ||
return 3000 | ||
|
||
|
||
def instance_disable_generic_tags(): | ||
return False | ||
|
||
|
||
def instance_empty_default_hostname(): | ||
return False | ||
|
||
|
||
def instance_min_collection_interval(): | ||
return 15 |
63 changes: 63 additions & 0 deletions
63
teleport/datadog_checks/teleport/config_models/instance.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,63 @@ | ||
# (C) Datadog, Inc. 2024-present | ||
# All rights reserved | ||
# Licensed under a 3-clause BSD style license (see LICENSE) | ||
|
||
# This file is autogenerated. | ||
# To change this file you should edit assets/configuration/spec.yaml and then run the following commands: | ||
# ddev -x validate config -s <INTEGRATION_NAME> | ||
# ddev -x validate models -s <INTEGRATION_NAME> | ||
|
||
from __future__ import annotations | ||
|
||
from typing import Optional | ||
|
||
from pydantic import BaseModel, ConfigDict, field_validator, model_validator | ||
|
||
from datadog_checks.base.utils.functions import identity | ||
from datadog_checks.base.utils.models import validation | ||
|
||
from . import defaults, validators | ||
|
||
|
||
class MetricPatterns(BaseModel): | ||
model_config = ConfigDict( | ||
arbitrary_types_allowed=True, | ||
frozen=True, | ||
) | ||
exclude: Optional[tuple[str, ...]] = None | ||
include: Optional[tuple[str, ...]] = None | ||
|
||
|
||
class InstanceConfig(BaseModel): | ||
model_config = ConfigDict( | ||
validate_default=True, | ||
arbitrary_types_allowed=True, | ||
frozen=True, | ||
) | ||
diag_port: Optional[int] = None | ||
disable_generic_tags: Optional[bool] = None | ||
empty_default_hostname: Optional[bool] = None | ||
metric_patterns: Optional[MetricPatterns] = None | ||
min_collection_interval: Optional[float] = None | ||
service: Optional[str] = None | ||
tags: Optional[tuple[str, ...]] = None | ||
teleport_url: str | ||
|
||
@model_validator(mode='before') | ||
def _initial_validation(cls, values): | ||
return validation.core.initialize_config(getattr(validators, 'initialize_instance', identity)(values)) | ||
|
||
@field_validator('*', mode='before') | ||
def _validate(cls, value, info): | ||
field = cls.model_fields[info.field_name] | ||
field_name = field.alias or info.field_name | ||
if field_name in info.context['configured_fields']: | ||
value = getattr(validators, f'instance_{info.field_name}', identity)(value, field=field) | ||
else: | ||
value = getattr(defaults, f'instance_{info.field_name}', lambda: value)() | ||
|
||
return validation.utils.make_immutable(value) | ||
|
||
@model_validator(mode='after') | ||
def _final_validation(cls, model): | ||
return validation.core.check_model(getattr(validators, 'check_instance', identity)(model)) |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.