YQ fixed parquet error description #19779
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR updates test configuration flags related to spilling behavior and improves the Parquet error message to include schema context for missing fields.
- Removed a deprecated write-portion flag and added three new spilling feature flags in the test config.
- Enhanced the
ParquetExceptionfor missing fields by listing the actual schema fields.
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| ydb/tests/tools/kqprun/configuration/app_config.conf | Removed EnableWritePortionsOnInsert and added new spilling feature flags |
| ydb/library/yql/providers/s3/actors/yql_arrow_column_converters.cpp | Expanded the missing-field exception message to include found fields |
Comments suppressed due to low confidence (2)
ydb/library/yql/providers/s3/actors/yql_arrow_column_converters.cpp:791
- Consider adding or updating a unit test to cover this new error path and verify that the exception message includes both the missing field name and the list of found schema fields.
throw parquet::ParquetException(TStringBuilder() << "Missing field: " << targetField->name() << ", found fields in arrow file: " << dataSchema->ToString());
ydb/tests/tools/kqprun/configuration/app_config.conf:220
- It would be helpful to add comments explaining the purpose and valid values for
EnableSpillingNodes(and the other new spilling flags) to guide future maintainers.
EnableSpillingNodes: "None"
|
🟢 |
|
⚪ Test history | Ya make output | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat | Test bloat
🟢
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
GrigoriyPA
added
the
rebase-and-check
github-actions
bot
removed
the
rebase-and-check
|
⚪ Test history | Ya make output | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat | Test bloat
🟢
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
|
⚪ Test history | Ya make output | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat | Test bloat
🟢
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
GrigoriyPA
added
the
rebase-and-check
github-actions
bot
removed
the
rebase-and-check
|
⚪ Test history | Ya make output | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat
🟢
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
|
⚪ Test history | Ya make output | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat | Test bloat
🟢
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
Changelog entry
Fixed parquet error description
Changelog category
Description for reviewers