Skip to content

Unparse of Joins is ignoring projections #15688

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
nuno-faria opened this issue Apr 11, 2025 · 1 comment · Fixed by #15693
Closed

Unparse of Joins is ignoring projections #15688

nuno-faria opened this issue Apr 11, 2025 · 1 comment · Fixed by #15693
Assignees
Labels
bug Something isn't working

Comments

@nuno-faria
Copy link
Contributor

Describe the bug

The unparse of Join operators is ignoring the projected columns, ending up projecting everything.

Two conditions cause this to happen:

  • the final projected columns match the columns in the TableScans, meaning the Projection is optimized away with the optimize_projections rule;
  • the try_transform_to_simple_table_scan_with_filters function in the unparser module -- used when processing joins -- causes the TableScan projection to be discarded.

I could try to fix this by modifying the join unparser, but I would like to be sure that this is the best approach here.

To Reproduce

use datafusion::error::Result;
use datafusion::prelude::SessionContext;
use datafusion::sql::unparser::Unparser;
use datafusion::sql::unparser::dialect::PostgreSqlDialect;

#[tokio::main]
async fn main() -> Result<()> {
    let ctx = SessionContext::new();

    ctx.sql("create table test (k int, v int)")
        .await?
        .collect()
        .await?;

    let df = ctx.sql("select t1.v, t2.v from test t1, test t2").await?;

    println!("{}\n", df.logical_plan());

    let plan = df.into_optimized_plan()?;
    println!("{}\n", plan);

    println!(
        "{}",
        Unparser::new(&PostgreSqlDialect {})
            .plan_to_sql(&plan)
            .unwrap()
    );

    Ok(())
}
Projection: t1.v, t2.v
  Cross Join: 
    SubqueryAlias: t1
      TableScan: test
    SubqueryAlias: t2
      TableScan: test

Cross Join:
  SubqueryAlias: t1
    TableScan: test projection=[v]
  SubqueryAlias: t2
    TableScan: test projection=[v]

SELECT * FROM "test" AS "t1" CROSS JOIN "test" AS "t2"

Expected behavior

Output the query with the projected fields:

SELECT "t1".v, "t2".v FROM "test" AS "t1" CROSS JOIN "test" AS "t2"

Additional context

No response

@nuno-faria nuno-faria added the bug Something isn't working label Apr 11, 2025
@chenkovsky
Copy link
Contributor

take

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants