Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feat][doc] PIP-264: Add replication metrics reference #926

Merged
merged 3 commits into from
Oct 29, 2024
Merged

[feat][doc] PIP-264: Add replication metrics reference #926

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
bd038c6
[feat][doc] PIP-264: Add replication metrics reference
dragosvictor Jun 25, 2024
a106df3
Merge remote-tracking branch 'origin/main' into dmisca-pip-264-replic…
lhotari Oct 29, 2024
dafd28c
Apply changes to 4.0.x
lhotari Oct 29, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
114 changes: 114 additions & 0 deletions docs/reference-metrics-opentelemetry.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -795,6 +795,120 @@ The current usage of threads in the pulsar-web executor pool.
* `current` - Indicates the total number of threads currently associated with the pool.
* `idle` - Indicates the number of threads available to serve requests.

### Replicator metrics

#### pulsar.broker.replication.message.incoming.count
The total number of messages received from the remote cluster through this replicator.
* Type: Counter
* Unit: `{message}`
* Attributes:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems like these attributes are copied multiple times. can we link a ref, instead?

* `pulsar.domain` - The domain of the topic. Can be one of:
* `persistent`
* `non-persistent`
* `pulsar.tenant` - The topic tenant.
* `pulsar.namespace` - The topic namespace.
* `pulsar.topic` - The topic name.
* `pulsar.partition.index` - The partition index of the topic. Present only if the topic is partitioned.
* `pulsar.replication.remote.cluster.name` - The name of the remote cluster.

#### pulsar.broker.replication.message.outgoing.count
The total number of messages sent to the remote cluster through this replicator.
* Type: Counter
* Unit: `{message}`
* Attributes:
* `pulsar.domain` - The domain of the topic. Can be one of:
* `persistent`
* `non-persistent`
* `pulsar.tenant` - The topic tenant.
* `pulsar.namespace` - The topic namespace.
* `pulsar.topic` - The topic name.
* `pulsar.partition.index` - The partition index of the topic. Present only if the topic is partitioned.
* `pulsar.replication.remote.cluster.name` - The name of the remote cluster.

#### pulsar.broker.replication.message.incoming.size
The total number of messages bytes received from the remote cluster through this replicator.
* Type: Counter
* Unit: `{By}`
* Attributes:
* `pulsar.domain` - The domain of the topic. Can be one of:
* `persistent`
* `non-persistent`
* `pulsar.tenant` - The topic tenant.
* `pulsar.namespace` - The topic namespace.
* `pulsar.topic` - The topic name.
* `pulsar.partition.index` - The partition index of the topic. Present only if the topic is partitioned.
* `pulsar.replication.remote.cluster.name` - The name of the remote cluster.

#### pulsar.broker.replication.message.outgoing.size
The total number of messages bytes sent to the remote cluster through this replicator.
* Type: Counter
* Unit: `{By}`
* Attributes:
* `pulsar.domain` - The domain of the topic. Can be one of:
* `persistent`
* `non-persistent`
* `pulsar.tenant` - The topic tenant.
* `pulsar.namespace` - The topic namespace.
* `pulsar.topic` - The topic name.
* `pulsar.partition.index` - The partition index of the topic. Present only if the topic is partitioned.
* `pulsar.replication.remote.cluster.name` - The name of the remote cluster.

#### pulsar.broker.replication.message.backlog.count
The total number of messages in the backlog for this replicator.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we clarify a little more here? something like

The total number of messages in the backlog for this replicator has not been consumed.

* Type: Counter
* Unit: `{message}`
* Attributes:
* `pulsar.domain` - The domain of the topic. Can be one of:
* `persistent`
* `non-persistent`
* `pulsar.tenant` - The topic tenant.
* `pulsar.namespace` - The topic namespace.
* `pulsar.topic` - The topic name.
* `pulsar.partition.index` - The partition index of the topic. Present only if the topic is partitioned.
* `pulsar.replication.remote.cluster.name` - The name of the remote cluster.

#### pulsar.broker.replication.message.backlog.age
The age of the oldest message in the replicator backlog.
* Type: Gauge
* Unit: `s`
* Attributes:
* `pulsar.domain` - The domain of the topic. Can be one of:
* `persistent`
* `non-persistent`
* `pulsar.tenant` - The topic tenant.
* `pulsar.namespace` - The topic namespace.
* `pulsar.topic` - The topic name.
* `pulsar.partition.index` - The partition index of the topic. Present only if the topic is partitioned.
* `pulsar.replication.remote.cluster.name` - The name of the remote cluster.

#### pulsar.broker.replication.message.expired.count
The total number of messages that expired for this replicator.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we clarify a little more here? something like

The total number of messages that expired for this replicator before consuming.

* Type: Counter
* Unit: `{message}`
* Attributes:
* `pulsar.domain` - The domain of the topic. Can be one of:
* `persistent`
* `non-persistent`
* `pulsar.tenant` - The topic tenant.
* `pulsar.namespace` - The topic namespace.
* `pulsar.topic` - The topic name.
* `pulsar.partition.index` - The partition index of the topic. Present only if the topic is partitioned.
* `pulsar.replication.remote.cluster.name` - The name of the remote cluster.

#### pulsar.broker.replication.message.dropped.count
The total number of messages dropped by this replicator.
* Type: Counter
* Unit: `{message}`
* Attributes:
* `pulsar.domain` - The domain of the topic. Can be one of:
* `persistent`
* `non-persistent`
* `pulsar.tenant` - The topic tenant.
* `pulsar.namespace` - The topic namespace.
* `pulsar.topic` - The topic name.
* `pulsar.partition.index` - The partition index of the topic. Present only if the topic is partitioned.
* `pulsar.replication.remote.cluster.name` - The name of the remote cluster.

### Schema Registry Metrics

#### pulsar.broker.request.schema_registry.duration
Expand Down
114 changes: 114 additions & 0 deletions versioned_docs/version-4.0.x/reference-metrics-opentelemetry.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -795,6 +795,120 @@ The current usage of threads in the pulsar-web executor pool.
* `current` - Indicates the total number of threads currently associated with the pool.
* `idle` - Indicates the number of threads available to serve requests.

### Replicator metrics

#### pulsar.broker.replication.message.incoming.count
The total number of messages received from the remote cluster through this replicator.
* Type: Counter
* Unit: `{message}`
* Attributes:
* `pulsar.domain` - The domain of the topic. Can be one of:
* `persistent`
* `non-persistent`
* `pulsar.tenant` - The topic tenant.
* `pulsar.namespace` - The topic namespace.
* `pulsar.topic` - The topic name.
* `pulsar.partition.index` - The partition index of the topic. Present only if the topic is partitioned.
* `pulsar.replication.remote.cluster.name` - The name of the remote cluster.

#### pulsar.broker.replication.message.outgoing.count
The total number of messages sent to the remote cluster through this replicator.
* Type: Counter
* Unit: `{message}`
* Attributes:
* `pulsar.domain` - The domain of the topic. Can be one of:
* `persistent`
* `non-persistent`
* `pulsar.tenant` - The topic tenant.
* `pulsar.namespace` - The topic namespace.
* `pulsar.topic` - The topic name.
* `pulsar.partition.index` - The partition index of the topic. Present only if the topic is partitioned.
* `pulsar.replication.remote.cluster.name` - The name of the remote cluster.

#### pulsar.broker.replication.message.incoming.size
The total number of messages bytes received from the remote cluster through this replicator.
* Type: Counter
* Unit: `{By}`
* Attributes:
* `pulsar.domain` - The domain of the topic. Can be one of:
* `persistent`
* `non-persistent`
* `pulsar.tenant` - The topic tenant.
* `pulsar.namespace` - The topic namespace.
* `pulsar.topic` - The topic name.
* `pulsar.partition.index` - The partition index of the topic. Present only if the topic is partitioned.
* `pulsar.replication.remote.cluster.name` - The name of the remote cluster.

#### pulsar.broker.replication.message.outgoing.size
The total number of messages bytes sent to the remote cluster through this replicator.
* Type: Counter
* Unit: `{By}`
* Attributes:
* `pulsar.domain` - The domain of the topic. Can be one of:
* `persistent`
* `non-persistent`
* `pulsar.tenant` - The topic tenant.
* `pulsar.namespace` - The topic namespace.
* `pulsar.topic` - The topic name.
* `pulsar.partition.index` - The partition index of the topic. Present only if the topic is partitioned.
* `pulsar.replication.remote.cluster.name` - The name of the remote cluster.

#### pulsar.broker.replication.message.backlog.count
The total number of messages in the backlog for this replicator.
* Type: Counter
* Unit: `{message}`
* Attributes:
* `pulsar.domain` - The domain of the topic. Can be one of:
* `persistent`
* `non-persistent`
* `pulsar.tenant` - The topic tenant.
* `pulsar.namespace` - The topic namespace.
* `pulsar.topic` - The topic name.
* `pulsar.partition.index` - The partition index of the topic. Present only if the topic is partitioned.
* `pulsar.replication.remote.cluster.name` - The name of the remote cluster.

#### pulsar.broker.replication.message.backlog.age
The age of the oldest message in the replicator backlog.
* Type: Gauge
* Unit: `s`
* Attributes:
* `pulsar.domain` - The domain of the topic. Can be one of:
* `persistent`
* `non-persistent`
* `pulsar.tenant` - The topic tenant.
* `pulsar.namespace` - The topic namespace.
* `pulsar.topic` - The topic name.
* `pulsar.partition.index` - The partition index of the topic. Present only if the topic is partitioned.
* `pulsar.replication.remote.cluster.name` - The name of the remote cluster.

#### pulsar.broker.replication.message.expired.count
The total number of messages that expired for this replicator.
* Type: Counter
* Unit: `{message}`
* Attributes:
* `pulsar.domain` - The domain of the topic. Can be one of:
* `persistent`
* `non-persistent`
* `pulsar.tenant` - The topic tenant.
* `pulsar.namespace` - The topic namespace.
* `pulsar.topic` - The topic name.
* `pulsar.partition.index` - The partition index of the topic. Present only if the topic is partitioned.
* `pulsar.replication.remote.cluster.name` - The name of the remote cluster.

#### pulsar.broker.replication.message.dropped.count
The total number of messages dropped by this replicator.
* Type: Counter
* Unit: `{message}`
* Attributes:
* `pulsar.domain` - The domain of the topic. Can be one of:
* `persistent`
* `non-persistent`
* `pulsar.tenant` - The topic tenant.
* `pulsar.namespace` - The topic namespace.
* `pulsar.topic` - The topic name.
* `pulsar.partition.index` - The partition index of the topic. Present only if the topic is partitioned.
* `pulsar.replication.remote.cluster.name` - The name of the remote cluster.

### Schema Registry Metrics

#### pulsar.broker.request.schema_registry.duration
Expand Down