-
Couldn't load subscription status.
- Fork 1.7k
fix: with_param_values on EmptyRelation returns incorrect schema
#18286
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
This reverts commit 9e1a473.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Makes sense to me 👍
Do you think it's also possible to add the queries from the original issue as tests?
This reverts commit f53cd3b.
|
Thanks @Jefffrey I tried to add test for the subquery alias: let df = ctx.sql("SELECT a, b FROM (VALUES ($1, $2)) AS t(a, b)").await?;
let df_with_params_replaced = df.with_param_values(vec![
ScalarValue::UInt32(Some(1)),
ScalarValue::Utf8(Some("foofy".to_string())),
])?;
dbg!(df_with_params_replaced.collect().await?[0].schema());
#> Error: ArrowError(InvalidArgumentError("column types must match schema types, expected Null but found UInt32 at column index 0"), Some(""))But the test still fail, the plan after replacing params (the schema for The expected params: I think it needs more works, convert to draft for now. |
Which issue does this PR close?
SubqueryAlias,Values, and/orEmptyRelationhave incorrect schemas after replacingPlaceholdervalues #18102.Rationale for this change
with_param_valuesdoesn't substitute params' type if it is used onEmptyRelation.Thus, causing
SELECT $1, $2to have incorrect schema after substitution.For example: after replacing
$1 = 1, $2 = "s", the schema is[Null, Null], but it shouldbe
[Int64, Utf8].This schema type mismatch is resolved before converting to physical plan by
the
type_coercionrule in theanalyzer.datafusion/datafusion/optimizer/src/analyzer/type_coercion.rs
Line 149 in 8142360
So I'm not quite sure should we fix it in
with_param_values.What changes are included in this PR?
Are these changes tested?
Yes.
Are there any user-facing changes?
No.