observability into refreshes

[nicks]

Describe the feature request

is there a way to get observability into how the proxy is refreshing?

Background

we had an incident a month ago where we thought it might be related to refreshing, and there wasn't an easy way to tell from the outside when the last time the proxy had refreshed its data

Solution suggestions

Logs would be fine (there are very few logs right now)

A prometheus-style metrics endpoint would also work.

[chriswk]

Hi @nicks . Thank you for your report.

By default, the proxy is setup to log errors if any happen, so if you're seeing a quiet log, it should mean you're not having a refreshing issue.

From our client constructor

        this.metrics.on('error', (msg) => this.logger.error(`metrics: ${msg}`));
        this.unleash.on('error', (msg) => this.logger.error(msg));

We could add two prometheus gauges, last_features_check_epoch_seconds and last_features_update_epoch_seconds to trigger on the events emitted by the UnleashClient when it does its poll cycle.

Unless you're using custom strategies or context enrichers, we do recommend swapping to Unleash Edge (https://github.com/unleash/unleash-edge) where we have an endpoint allowing you to see the status of all client keys being used.

[chriswk]

Added and released as v1.4.5 of the proxy, which should be available in the coming 20 minutes or so.

[chriswk]

Copying from the README what we added.

Prometheus endpoint

The proxy has a prometheus metrics endpoint available at http://localhost:3000/proxy/internal-backstage/prometheus With the following metrics

    unleash_proxy_up a [counter](https://prometheus.io/docs/concepts/metric_types/#counter) which is set to 1 when proxy is running
    last_metrics_update_epoch_timestamp_ms a [gauge](https://prometheus.io/docs/concepts/metric_types/#gauge) set to the epoch timestamp in ms when the proxy last received a feature update
    last_metrics_fetch_epoch_timestamp_ms a [gauge](https://prometheus.io/docs/concepts/metric_types/#gauge) set to the epoch timestamp in ms when the proxy last checked for updates