From bd038c6b87fd883b78cb7dc60cea721152945e94 Mon Sep 17 00:00:00 2001 From: Dragos Misca Date: Mon, 24 Jun 2024 21:06:12 -0700 Subject: [PATCH 1/2] [feat][doc] PIP-264: Add replication metrics reference --- docs/reference-metrics-opentelemetry.md | 114 ++++++++++++++++++++++++ 1 file changed, 114 insertions(+) diff --git a/docs/reference-metrics-opentelemetry.md b/docs/reference-metrics-opentelemetry.md index 4f337fcbbf36..39297912ac8d 100644 --- a/docs/reference-metrics-opentelemetry.md +++ b/docs/reference-metrics-opentelemetry.md @@ -478,6 +478,120 @@ The total number of messages dropped from this producer. * `wait_for_exclusive` * `exclusive_with_fencing` +### Replicator metrics + +#### pulsar.broker.replication.message.incoming.count +The total number of messages received from the remote cluster through this replicator. +* Type: Counter +* Unit: `{message}` +* Attributes: + * `pulsar.domain` - The domain of the topic. Can be one of: + * `persistent` + * `non-persistent` + * `pulsar.tenant` - The topic tenant. + * `pulsar.namespace` - The topic namespace. + * `pulsar.topic` - The topic name. + * `pulsar.partition.index` - The partition index of the topic. Present only if the topic is partitioned. + * `pulsar.replication.remote.cluster.name` - The name of the remote cluster. + +#### pulsar.broker.replication.message.outgoing.count +The total number of messages sent to the remote cluster through this replicator. +* Type: Counter +* Unit: `{message}` +* Attributes: + * `pulsar.domain` - The domain of the topic. Can be one of: + * `persistent` + * `non-persistent` + * `pulsar.tenant` - The topic tenant. + * `pulsar.namespace` - The topic namespace. + * `pulsar.topic` - The topic name. + * `pulsar.partition.index` - The partition index of the topic. Present only if the topic is partitioned. + * `pulsar.replication.remote.cluster.name` - The name of the remote cluster. + +#### pulsar.broker.replication.message.incoming.size +The total number of messages bytes received from the remote cluster through this replicator. +* Type: Counter +* Unit: `{By}` +* Attributes: + * `pulsar.domain` - The domain of the topic. Can be one of: + * `persistent` + * `non-persistent` + * `pulsar.tenant` - The topic tenant. + * `pulsar.namespace` - The topic namespace. + * `pulsar.topic` - The topic name. + * `pulsar.partition.index` - The partition index of the topic. Present only if the topic is partitioned. + * `pulsar.replication.remote.cluster.name` - The name of the remote cluster. + +#### pulsar.broker.replication.message.outgoing.size +The total number of messages bytes sent to the remote cluster through this replicator. +* Type: Counter +* Unit: `{By}` +* Attributes: + * `pulsar.domain` - The domain of the topic. Can be one of: + * `persistent` + * `non-persistent` + * `pulsar.tenant` - The topic tenant. + * `pulsar.namespace` - The topic namespace. + * `pulsar.topic` - The topic name. + * `pulsar.partition.index` - The partition index of the topic. Present only if the topic is partitioned. + * `pulsar.replication.remote.cluster.name` - The name of the remote cluster. + +#### pulsar.broker.replication.message.backlog.count +The total number of messages in the backlog for this replicator. +* Type: Counter +* Unit: `{message}` +* Attributes: + * `pulsar.domain` - The domain of the topic. Can be one of: + * `persistent` + * `non-persistent` + * `pulsar.tenant` - The topic tenant. + * `pulsar.namespace` - The topic namespace. + * `pulsar.topic` - The topic name. + * `pulsar.partition.index` - The partition index of the topic. Present only if the topic is partitioned. + * `pulsar.replication.remote.cluster.name` - The name of the remote cluster. + +#### pulsar.broker.replication.message.backlog.age +The age of the oldest message in the replicator backlog. +* Type: Gauge +* Unit: `s` +* Attributes: + * `pulsar.domain` - The domain of the topic. Can be one of: + * `persistent` + * `non-persistent` + * `pulsar.tenant` - The topic tenant. + * `pulsar.namespace` - The topic namespace. + * `pulsar.topic` - The topic name. + * `pulsar.partition.index` - The partition index of the topic. Present only if the topic is partitioned. + * `pulsar.replication.remote.cluster.name` - The name of the remote cluster. + +#### pulsar.broker.replication.message.expired.count +The total number of messages that expired for this replicator. +* Type: Counter +* Unit: `{message}` +* Attributes: + * `pulsar.domain` - The domain of the topic. Can be one of: + * `persistent` + * `non-persistent` + * `pulsar.tenant` - The topic tenant. + * `pulsar.namespace` - The topic namespace. + * `pulsar.topic` - The topic name. + * `pulsar.partition.index` - The partition index of the topic. Present only if the topic is partitioned. + * `pulsar.replication.remote.cluster.name` - The name of the remote cluster. + +#### pulsar.broker.replication.message.dropped.count +The total number of messages dropped by this replicator. +* Type: Counter +* Unit: `{message}` +* Attributes: + * `pulsar.domain` - The domain of the topic. Can be one of: + * `persistent` + * `non-persistent` + * `pulsar.tenant` - The topic tenant. + * `pulsar.namespace` - The topic namespace. + * `pulsar.topic` - The topic name. + * `pulsar.partition.index` - The partition index of the topic. Present only if the topic is partitioned. + * `pulsar.replication.remote.cluster.name` - The name of the remote cluster. + ### Schema Registry Metrics #### pulsar.broker.request.schema_registry.duration From dafd28cb875bef4c91414784bd5f19a2bcc1b8c2 Mon Sep 17 00:00:00 2001 From: Lari Hotari Date: Tue, 29 Oct 2024 13:00:35 +0200 Subject: [PATCH 2/2] Apply changes to 4.0.x --- .../reference-metrics-opentelemetry.md | 114 ++++++++++++++++++ 1 file changed, 114 insertions(+) diff --git a/versioned_docs/version-4.0.x/reference-metrics-opentelemetry.md b/versioned_docs/version-4.0.x/reference-metrics-opentelemetry.md index 4d21357bf5e5..3594dc57ce61 100644 --- a/versioned_docs/version-4.0.x/reference-metrics-opentelemetry.md +++ b/versioned_docs/version-4.0.x/reference-metrics-opentelemetry.md @@ -795,6 +795,120 @@ The current usage of threads in the pulsar-web executor pool. * `current` - Indicates the total number of threads currently associated with the pool. * `idle` - Indicates the number of threads available to serve requests. +### Replicator metrics + +#### pulsar.broker.replication.message.incoming.count +The total number of messages received from the remote cluster through this replicator. +* Type: Counter +* Unit: `{message}` +* Attributes: + * `pulsar.domain` - The domain of the topic. Can be one of: + * `persistent` + * `non-persistent` + * `pulsar.tenant` - The topic tenant. + * `pulsar.namespace` - The topic namespace. + * `pulsar.topic` - The topic name. + * `pulsar.partition.index` - The partition index of the topic. Present only if the topic is partitioned. + * `pulsar.replication.remote.cluster.name` - The name of the remote cluster. + +#### pulsar.broker.replication.message.outgoing.count +The total number of messages sent to the remote cluster through this replicator. +* Type: Counter +* Unit: `{message}` +* Attributes: + * `pulsar.domain` - The domain of the topic. Can be one of: + * `persistent` + * `non-persistent` + * `pulsar.tenant` - The topic tenant. + * `pulsar.namespace` - The topic namespace. + * `pulsar.topic` - The topic name. + * `pulsar.partition.index` - The partition index of the topic. Present only if the topic is partitioned. + * `pulsar.replication.remote.cluster.name` - The name of the remote cluster. + +#### pulsar.broker.replication.message.incoming.size +The total number of messages bytes received from the remote cluster through this replicator. +* Type: Counter +* Unit: `{By}` +* Attributes: + * `pulsar.domain` - The domain of the topic. Can be one of: + * `persistent` + * `non-persistent` + * `pulsar.tenant` - The topic tenant. + * `pulsar.namespace` - The topic namespace. + * `pulsar.topic` - The topic name. + * `pulsar.partition.index` - The partition index of the topic. Present only if the topic is partitioned. + * `pulsar.replication.remote.cluster.name` - The name of the remote cluster. + +#### pulsar.broker.replication.message.outgoing.size +The total number of messages bytes sent to the remote cluster through this replicator. +* Type: Counter +* Unit: `{By}` +* Attributes: + * `pulsar.domain` - The domain of the topic. Can be one of: + * `persistent` + * `non-persistent` + * `pulsar.tenant` - The topic tenant. + * `pulsar.namespace` - The topic namespace. + * `pulsar.topic` - The topic name. + * `pulsar.partition.index` - The partition index of the topic. Present only if the topic is partitioned. + * `pulsar.replication.remote.cluster.name` - The name of the remote cluster. + +#### pulsar.broker.replication.message.backlog.count +The total number of messages in the backlog for this replicator. +* Type: Counter +* Unit: `{message}` +* Attributes: + * `pulsar.domain` - The domain of the topic. Can be one of: + * `persistent` + * `non-persistent` + * `pulsar.tenant` - The topic tenant. + * `pulsar.namespace` - The topic namespace. + * `pulsar.topic` - The topic name. + * `pulsar.partition.index` - The partition index of the topic. Present only if the topic is partitioned. + * `pulsar.replication.remote.cluster.name` - The name of the remote cluster. + +#### pulsar.broker.replication.message.backlog.age +The age of the oldest message in the replicator backlog. +* Type: Gauge +* Unit: `s` +* Attributes: + * `pulsar.domain` - The domain of the topic. Can be one of: + * `persistent` + * `non-persistent` + * `pulsar.tenant` - The topic tenant. + * `pulsar.namespace` - The topic namespace. + * `pulsar.topic` - The topic name. + * `pulsar.partition.index` - The partition index of the topic. Present only if the topic is partitioned. + * `pulsar.replication.remote.cluster.name` - The name of the remote cluster. + +#### pulsar.broker.replication.message.expired.count +The total number of messages that expired for this replicator. +* Type: Counter +* Unit: `{message}` +* Attributes: + * `pulsar.domain` - The domain of the topic. Can be one of: + * `persistent` + * `non-persistent` + * `pulsar.tenant` - The topic tenant. + * `pulsar.namespace` - The topic namespace. + * `pulsar.topic` - The topic name. + * `pulsar.partition.index` - The partition index of the topic. Present only if the topic is partitioned. + * `pulsar.replication.remote.cluster.name` - The name of the remote cluster. + +#### pulsar.broker.replication.message.dropped.count +The total number of messages dropped by this replicator. +* Type: Counter +* Unit: `{message}` +* Attributes: + * `pulsar.domain` - The domain of the topic. Can be one of: + * `persistent` + * `non-persistent` + * `pulsar.tenant` - The topic tenant. + * `pulsar.namespace` - The topic namespace. + * `pulsar.topic` - The topic name. + * `pulsar.partition.index` - The partition index of the topic. Present only if the topic is partitioned. + * `pulsar.replication.remote.cluster.name` - The name of the remote cluster. + ### Schema Registry Metrics #### pulsar.broker.request.schema_registry.duration