fix(cosmos-vstorage): split 'size_delta' counter into 'size_increase' and 'size_decrease'

[siarhei-agoric]

refs: #11062
refs: #10938

Description

Despite testing, telemetry.IncrCounterWithLabels() appears to only accept non-negative numbers. As a workaround, we will be reporting vstorage size increase and decrease as two separate counters.

Security Considerations

N/A

Scaling Considerations

One extra metric not an issue as long as number of keeper instances is low.

Documentation Considerations

New metrics:

# HELP store_size_increase store_size_increase
# TYPE store_size_increase counter
store_size_increase{storeKey="vstorage"} 42386

# HELP store_size_decrease store_size_decrease
# TYPE store_size_decrease counter
store_size_decrease{storeKey="vstorage"} 335

Testing Considerations

Manual Testing; Automated testing will be submitted as a separate PR.

First terminal:

cd agoric-sdk/packages/cosmic-swingset
make scenario2-setup scenario2-run-chain

Second terminal (after Block 17)

cd agoric-sdk/packages/cosmic-swingset
make fund-provision-pool
make provision-acct ACCT_ADDR=$(cat t1/8000/ag-cosmos-helper-address)

Third terminal (start before Block 17):

cd agoric-sdk/packages/cosmic-swingset
while true; do curl -s -G 'http://localhost:26660/metrics' | grep 'store_size_'; sleep 10; done
[...]
# HELP store_size_increase store_size_increase
# TYPE store_size_increase counter
store_size_increase{storeKey="vstorage"} 42386
# HELP store_size_decrease store_size_decrease
# TYPE store_size_decrease counter
store_size_decrease{storeKey="vstorage"} 335
# HELP store_size_increase store_size_increase
# TYPE store_size_increase counter
store_size_increase{storeKey="vstorage"} 284
# HELP store_size_decrease store_size_decrease
# TYPE store_size_decrease counter
store_size_decrease{storeKey="vstorage"} 335
# HELP store_size_increase store_size_increase
# TYPE store_size_increase counter
store_size_increase{storeKey="vstorage"} 284
# HELP store_size_decrease store_size_decrease
# TYPE store_size_decrease counter
store_size_decrease{storeKey="vstorage"} 788

Upgrade Considerations

N/A

[gibson042]

The changes look good, but blocking until we have test coverage.

[cloudflare-workers-and-pages]

Deploying agoric-sdk with    Cloudflare Pages

Latest commit:	513ee68
Status:	✅  Deploy successful!
Preview URL:	https://f715145d.agoric-sdk.pages.dev
Branch Preview URL:	https://sliakh-11062-agd-panic-vstor.agoric-sdk.pages.dev

View logs

[gibson042]

Unblocking per the recorded manual testing and with the expectation of automated testing in a followup.

[siarhei-agoric]

The changes look good, but blocking until we have test coverage.

I'd like to have this merged to get master fixed ASAP with just manual testing. A separate PR will be created for the automated testing.

[michaelfig]

Why not make this a method on the Keeper?

Suggested change

	func ReportStoreSizeMetrics(k Keeper, sizeDelta float32) {
	// ReportStoreSizeMetrics exports sizeDelta metrics when
	// Cosmos telemetry is enabled.
	func (k Keeper) ReportStoreSizeMetrics(sizeDelta float32) {

[mhofman]

Since we're reporting 2 separate metrics, I would have preferred reporting a down of the current size and an up of the new size instead of a single up/down depending on the direction of the change. That gives us a little more information.

[mhofman]

Suggested change

	telemetry.IncrCounterWithLabels(MetricKeyStoreSizeDecrese, -sizeDelta, metricsLabel)
	telemetry.IncrCounterWithLabels(MetricKeyStoreSizeDecrease, -sizeDelta, metricsLabel)