Skip to content

Provide bandwidth metrics for 1-1-restore (similarly to regular L&S restore) #4373

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
mikliapko opened this issue May 5, 2025 · 2 comments
Open

Provide bandwidth metrics for 1-1-restore (similarly to regular L&S restore) #4373

mikliapko opened this issue May 5, 2025 · 2 comments
Assignees
@VAveryanov8
Labels
area/backup and restore
Milestone
1-1 Restore

Comments

@mikliapko
Copy link

mikliapko commented May 5, 2025 •
edited
Loading

Regular L&S restore provides bandwidth characteristics of the process in MiB/s/shard.

Thus, we are able to compare restore process speed across different backup sizes and instance types.

To follow the same goal, it'd be nice to have the same metrics for 1-1-restore.
@mikliapko mikliapko added this to the 1-1 Restore milestone May 5, 2025
@Michal-Leszczynski
Copy link
Collaborator

@mikliapko is this a dup of #4204?

VAveryanov8 added a commit that referenced this issue May 13, 2025
@VAveryanov8
feat(1-1-restore): add download bandwidth to logs
a6be9a1 
This introduces restoreStats struct which can be used to collect various
information statistics during 1-1-restore run.
For now main use case is to provide avg download bandwidth in the same
format as regular restore task progress, so SCT (e2e tests) can use it
to compare regular and 1-1 restores.
Also this adds bandwidth per table in bytes/ms which can be useful for
investigative purposes.

Refs: #4373
VAveryanov8 added a commit that referenced this issue May 13, 2025
@VAveryanov8
feat(1-1-restore): add download bandwidth to logs
a2ca922 
This introduces restoreStats struct which can be used to collect various
information statistics during 1-1-restore run.
For now main use case is to provide avg download bandwidth in the same
format as regular restore task progress, so SCT (e2e tests) can use it
to compare regular and 1-1 restores.
Also this adds bandwidth per table in bytes/ms which can be useful for
investigative purposes.

Refs: #4373
@VAveryanov8 VAveryanov8 mentioned this issue May 13, 2025
Merged
VAveryanov8 added a commit that referenced this issue May 13, 2025
@VAveryanov8
feat(1-1-restore): add download bandwidth to logs
97a5c12 
This introduces restoreStats struct which can be used to collect various
information statistics during 1-1-restore run.
For now main use case is to provide avg download bandwidth in the same
format as regular restore task progress, so SCT (e2e tests) can use it
to compare regular and 1-1 restores.
Also this adds bandwidth per table in bytes/ms which can be useful for
investigative purposes.

Refs: #4373
VAveryanov8 added a commit that referenced this issue May 29, 2025
@VAveryanov8
feat(1-1-restore): add download bandwidth to logs
9755d06 
This introduces restoreStats struct which can be used to collect various
information statistics during 1-1-restore run.
For now main use case is to provide avg download bandwidth in the same
format as regular restore task progress, so SCT (e2e tests) can use it
to compare regular and 1-1 restores.
Also this adds bandwidth per table in bytes/ms which can be useful for
investigative purposes.

Refs: #4373
VAveryanov8 added a commit that referenced this issue May 29, 2025
@VAveryanov8
feat(1-1-restore): add download bandwidth to logs
8563306 
This introduces restoreStats struct which can be used to collect various
information statistics during 1-1-restore run.
For now main use case is to provide avg download bandwidth in the same
format as regular restore task progress, so SCT (e2e tests) can use it
to compare regular and 1-1 restores.
Also this adds bandwidth per table in bytes/ms which can be useful for
investigative purposes.

Refs: #4373
@VAveryanov8
Copy link
Collaborator

update: instead of providing bandwidth to progress output it was decided to just log it and tests should be able to extract the value from the log. Here is an example of log record with bandwidth:

11:18:09.464    INFO    one2onerestore  Restore stats   {"bandwidth_per_shard": "10.540KiB/s/shard", "_trace_id": "SPYCOXJSSP6bxztF_Rh2IQ"}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@VAveryanov8 VAveryanov8

Labels
area/backup and restore
Projects
None yet
Milestone
1-1 Restore
Development

No branches or pull requests
4 participants
@VAveryanov8 @karol-kokoszka @mikliapko @Michal-Leszczynski