Unnested fields are not filterable when using subqueries. #16695
Description
Describe the bug
When attempting to evaluate a logical plan which mixes un-nesting and sub-queries, an unfiltered call to show will return the full set of columns, but attempting to filter on any column returned yields the following error:
Error: SchemaError(FieldNotFound { field: Column { relation: None, name: "__unnest_placeholder(one.__unnest_placeholder(mock.metadata).product).name" }, valid_fields: [Column { relation: Some(Bare { table: "mock" }), name: "metadata" }] }, Some(""))
To Reproduce
With nested.ndjson as:
{"metadata": {"product": {"name": "Product Name"}}}
The following test case reproduces the error:
use datafusion::common::DataFusionError;
use datafusion::prelude::{NdJsonReadOptions, SessionContext};
#[tokio::test]
async fn test_bug() -> Result<(), DataFusionError> {
let ctx = SessionContext::new();
ctx.register_json(
"mock",
"tests/fixtures/nested.ndjson",
NdJsonReadOptions::default().file_extension("ndjson")
).await?;
let query = r#"
WITH one AS (SELECT unnest(metadata) FROM mock),
two AS (SELECT unnest("__unnest_placeholder(mock.metadata).product") FROM one)
SELECT * FROM two WHERE "__unnest_placeholder(one.__unnest_placeholder(mock.metadata).product).name" == 'Product Name'
"#;
let frame = ctx.sql(query).await?;
frame.show().await?;
Ok(())
}Expected behavior
I would expect that the query completes and applies the filter.
Additional context
No response