Commit a9f316b
authored
Use in-memory buffer for arrow_writer benchmark (#7823)
# Which issue does this PR close?
Prerequisite for investigating parquet writing performance (#7822).
# Rationale for this change
The benchmark should measure the cpu overhead of parquet writing, not
the os or filesystem parts of it. Running the benchmark showed that the
file has nearly a 50% overhead, which makes profiling more difficult by
hiding the bottlenecks inside the parquet code itself.
# What changes are included in this PR?
Use a Vec instead of an unbuffered File as the sink.
# Are these changes tested?
Tested by running the benchmark.
# Are there any user-facing changes?
No1 parent 19a14dc commit a9f316b
1 file changed
+2
-5
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
19 | 19 | | |
20 | 20 | | |
21 | 21 | | |
22 | | - | |
23 | | - | |
24 | 22 | | |
25 | 23 | | |
26 | 24 | | |
| |||
349 | 347 | | |
350 | 348 | | |
351 | 349 | | |
352 | | - | |
353 | | - | |
354 | | - | |
| 350 | + | |
| 351 | + | |
355 | 352 | | |
356 | 353 | | |
357 | 354 | | |
| |||
0 commit comments