-
Notifications
You must be signed in to change notification settings - Fork 3.3k
Improve performance of PagesIndexComparator for strings #26725
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR improves the performance of ORDER BY queries by optimizing string comparisons in PagesIndexComparator
. The key improvement is switching from using BLOCK_POSITION
to BLOCK_POSITION_NOT_NULL
arguments in ordering operators, which allows the system to use a more efficient comparison method that avoids unnecessary Slice copies for string types.
- Updated ordering operator generation to use
BLOCK_POSITION_NOT_NULL
instead ofBLOCK_POSITION
for better performance - Added scoring logic for the new
VALUE_BLOCK_POSITION_NOT_NULL
argument convention - Enhanced benchmark tests to measure performance improvements for both BIGINT and VARCHAR types
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
File | Description |
---|---|
TypeOperators.java | Modified ordering operator generation to use more efficient comparison operators and added scoring for new argument convention |
BenchmarkPagesIndexOrdering.java | Enhanced benchmark to test both BIGINT and VARCHAR types with updated timing parameters |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
core/trino-spi/src/main/java/io/trino/spi/type/TypeOperators.java
Outdated
Show resolved
Hide resolved
42a029f
to
42ba088
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
42ba088
to
44cc12f
Compare
core/trino-spi/src/main/java/io/trino/spi/type/TypeOperators.java
Outdated
Show resolved
Hide resolved
Made changes to prefer adapting ordering operator from the more efficient version of DefaultOrderingOperators#comparisonOperator that avoids unnecessary copies of Slice for comparison BenchmarkPagesIndexOrdering.benchmarkQuickSort (numberOfChannels) (typeName) Mode Cnt Before score After score Unit 1 BIGINT avgt 20 29.536 ± 0.696 28.299 ± 0.344 ms/op 1 VARCHAR avgt 20 66.897 ± 1.914 53.538 ± 0.635 ms/op 10 BIGINT avgt 20 36.965 ± 0.270 36.577 ± 0.605 ms/op 10 VARCHAR avgt 20 79.887 ± 3.077 60.413 ± 0.678 ms/op
44cc12f
to
a5f288e
Compare
Started benchmark workflow for this PR with test type =
|
Started benchmark workflow for this PR with test type =
|
Started benchmark workflow for this PR with test type =
|
Description
Made changes to prefer adapting ordering operator from the
more efficient version of
DefaultOrderingOperators#comparisonOperator
that avoids unnecessary copies of Slice for comparison
E.g. from current code
Now this will use
io.trino.spi.type.AbstractVariableWidthType.DefaultOrderingOperators#comparisonOperator(io.trino.spi.block.VariableWidthBlock, int, io.trino.spi.block.VariableWidthBlock, int)
instead which avoids Slice copiesAdditional context and related issues
Release notes
( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
(x) Release notes are required, with the following suggested text: