-
Notifications
You must be signed in to change notification settings - Fork 199
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feat: improve CLI evaluation progress bar #4000
Open
treysp
wants to merge
18
commits into
main
Choose a base branch
from
trey/eval-prog-bar
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
+323
−119
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
1119862
to
e76ebb8
Compare
erindru
reviewed
Mar 14, 2025
fddbf4d
to
b5779b0
Compare
cf879e3
to
f9310ad
Compare
izeigerman
reviewed
Mar 28, 2025
izeigerman
reviewed
Mar 28, 2025
izeigerman
approved these changes
Mar 28, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please consider my suggestions. Looks good otherwise
82c06ff
to
9b26ec2
Compare
3eb3ccc
to
708b301
Compare
708b301
to
ec659e4
Compare
4b0420b
to
0cff15f
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR updates the CLI
plan
model evaluation progress bar output and modifies all progress bar labels.Evaluation progress bar
The evaluation progress bar prints one line of output for each executed batch.
Currently, it includes the node's batch number, node's name, and batch run time:
[3/7] sushi.customer_revenue_lifetime evaluated in 0.02s
The new output includes more information for each batch and is vertically aligned across batches. The column widths are static and specified in the code. If a model's information string is longer than the column width, the information is truncated with 3 dots (dots on left for node names so you see table, right for annotation so you see SQLMesh action).
This annotated screenshot below shows evaluation progress bar output from a modified version of the sushi project. The output includes an annotation of what SQLMesh did for each node:
raw.demographics
is an external model, and SQLMesh ran its auditssushi.added_model
is an hourly incremental by time model processing an interval <24h, so its interval start/end times are displayedsushi.raw_marketing
is a full model, so SQLMesh fully refreshed the tablesushi.waiter_names
is a seed model, so SQLMesh inserted from fileassert_item_price_above_zero
is a standalone audit, which SQLMesh ransushi.top_waiters
is a view, so SQLMesh recreated the viewModel kind annotations not in screenshot:
INCREMENTAL_BY_UNIQUE_KEY
: [insert or update rows]INCREMENTAL_BY_PARTITION
: [insert partition]All other model kinds display the processed interval start/ends, including time if the interval is <24h long.
Progress bar language
Creating physical tables
-->Updating physical layer
Model versions created successfully
-->✔ Physical layer updated
Evaluating models
-->Executing model batches
Model batches executed successfully
-->✔ Model batches executed
Virtually Updating 'prod'
-->Updating virtual layer
Target environment updated successfully
-->✔ Virtual layer updated
Default catalog in display names
Currently, we only display a model's catalog if it was explicitly specified (and not if SQLMesh used the default catalog).
This PR updates very verbose mode (
sqlmesh plan -vv
) to include default catalog in all model names.