Skip to content

Commit 50e6114

Browse files
Update upgrading.md for new unified config for sql string mapping to utf8view (#16809)
* Update upgrade md for new unified config for sql string mapping to utf8view * fix formtat * fix * format * Update docs/source/library-user-guide/upgrading.md --------- Co-authored-by: Oleks V <[email protected]>
1 parent 44e63e0 commit 50e6114

File tree

1 file changed

+50
-0
lines changed

1 file changed

+50
-0
lines changed

docs/source/library-user-guide/upgrading.md

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -120,6 +120,56 @@ SET datafusion.execution.spill_compression = 'zstd';
120120

121121
For more details about this configuration option, including performance trade-offs between different compression codecs, see the [Configuration Settings](../user-guide/configs.md) documentation.
122122

123+
### Deprecated `map_varchar_to_utf8view` configuration option
124+
125+
See [issue #16290](https://github.com/apache/datafusion/pull/16290) for more information
126+
The old configuration
127+
128+
```text
129+
datafusion.sql_parser.map_varchar_to_utf8view
130+
```
131+
132+
is now **deprecated** in favor of the unified option below.\
133+
If you previously used this to control only `VARCHAR``Utf8View` mapping, please migrate to `map_string_types_to_utf8view`.
134+
135+
---
136+
137+
### New `map_string_types_to_utf8view` configuration option
138+
139+
To unify **all** SQL string types (`CHAR`, `VARCHAR`, `TEXT`, `STRING`) to Arrow’s zero‑copy `Utf8View`, DataFusion 49.0.0 introduces:
140+
141+
- **Key**: `datafusion.sql_parser.map_string_types_to_utf8view`
142+
- **Default**: `true`
143+
144+
**Description:**
145+
146+
- When **true** (default), **all** SQL string types are mapped to `Utf8View`, avoiding full‑copy UTF‑8 allocations and improving performance.
147+
- When **false**, DataFusion falls back to the legacy `Utf8` mapping for **all** string types.
148+
149+
#### Examples
150+
151+
```rust
152+
# /* comment to avoid running
153+
// Disable Utf8View mapping for all SQL string types
154+
let opts = datafusion::sql::planner::ParserOptions::new()
155+
.with_map_string_types_to_utf8view(false);
156+
157+
// Verify the setting is applied
158+
assert!(!opts.map_string_types_to_utf8view);
159+
# */
160+
```
161+
162+
---
163+
164+
```sql
165+
-- Disable Utf8View mapping globally
166+
SET datafusion.sql_parser.map_string_types_to_utf8view = false;
167+
168+
-- Now VARCHAR, CHAR, TEXT, STRING all use Utf8 rather than Utf8View
169+
CREATE TABLE my_table (a VARCHAR, b TEXT, c STRING);
170+
DESCRIBE my_table;
171+
```
172+
123173
### Deprecating `SchemaAdapterFactory` and `SchemaAdapter`
124174

125175
We are moving away from converting data (using `SchemaAdapter`) to converting the expressions themselves (which is more efficient and flexible).

0 commit comments

Comments
 (0)