Skip to content

Conversation

klion26
Copy link
Member

@klion26 klion26 commented Aug 11, 2025

Which issue does this PR close?

We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax.

Rationale for this change

Implement VariantArray::value for some more shredded variants(eg. primitive_conversion/generic_conversion/non_generic_conversion).

What changes are included in this PR?

  • Extract all macroRules to a separate module type_conversion.rs
  • Add a macro for variant value

Are these changes tested?

Covered by the existing test

Are there any user-facing changes?

No

@klion26
Copy link
Member Author

klion26 commented Aug 11, 2025

@alamb I've drafted a PR for #8091 to confirm if this is the right direction; if so, I'll continue the implementation; if not, I can revise it

}};
}
//
// /// Convert the input array to a `VariantArray` row by row, using `method`
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment out this for the draft version; I will remove it in the future.

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @klion26 -- this is very cool. I am sorry for the delay in reviewing. I left some comments -- let me know what you think


/// Convert the input array to a `VariantArray` row by row, using `method`
/// to downcast the generic array to a specific array type and `cast_fn`
/// to transform each element to a type compatible with Variant
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is an interesting idea -- to have two variations of each macro. It is probably a reasonable place to start

However, I think it would be nice to avoid having macros in the root lib.rs file -- what do you think about making a new module like parquet-variant-compute/src/arrow_types.rs or parquet-variant-compute/src/type_conversion.rs to hold the code to convert back and forth between Arrow types and Variant types

@scovich added similar code in VariantToPrimitive here: #8122

So I am thinking having a module to start collecting such code would be useful

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Place the macros in parquet-variant-compute/src/type_conversion.rs is better, will check #8122 and fix the remainings

let typed_value = typed_value.as_primitive::<Int32Type>();
Variant::from(typed_value.value(index))
}
DataType::Int16 => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good -- we can probably change the DataType::Int32 branch to use this macro too, right?

Also, I think we should add a test

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, only Int16 is implemented to confirm the direction

Indeed, we need to add tests to cover this logic.

Copy link
Member Author

@klion26 klion26 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alamb Thanks for the review; I'll refine the PR and contact you after I'm done.


/// Convert the input array to a `VariantArray` row by row, using `method`
/// to downcast the generic array to a specific array type and `cast_fn`
/// to transform each element to a type compatible with Variant
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Place the macros in parquet-variant-compute/src/type_conversion.rs is better, will check #8122 and fix the remainings

let typed_value = typed_value.as_primitive::<Int32Type>();
Variant::from(typed_value.value(index))
}
DataType::Int16 => {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, only Int16 is implemented to confirm the direction

Indeed, we need to add tests to cover this logic.

@alamb alamb marked this pull request as draft August 14, 2025 16:58
@alamb
Copy link
Contributor

alamb commented Aug 14, 2025

marking as draft so it is clear this one isn't waiting on a review

@klion26 klion26 force-pushed the variant_value_get branch from caa9420 to d682bfd Compare August 18, 2025 01:03
@github-actions github-actions bot added the parquet-variant parquet-variant* crates label Aug 18, 2025
@klion26 klion26 force-pushed the variant_value_get branch 2 times, most recently from 19e0b2b to 363f9ad Compare August 18, 2025 01:15
@klion26 klion26 marked this pull request as ready for review August 18, 2025 01:24
@klion26
Copy link
Member Author

klion26 commented Aug 18, 2025

@alamb, please help review this when you have time. Thanks. This is ready to review now.


/// Convert the value at a specific index in the given array into a `Variant`.
#[macro_export]
macro_rules! primitive_conversion_single_value {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed to a separate macro other than a macro rules in the same macro as before, I think the current implementation is better for the user to choose which one they need.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

I was thinking about if there is some way to avoid defining 2 macros for each type of array, but given the branches are different (array.is_null --> null or Variant::Null ) I suspect the complexity of refactoring required would outweight any "don't repeat yourself" benefits

as_type,
cast_options,
))),
DataType::Int16 => Ok(Box::new(PrimitiveOutputBuilder::<Int16Type>::new(
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will add the remaining types in a follow-up pr

@klion26 klion26 requested a review from alamb August 18, 2025 04:47
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @klion26 -- I think this makes sense to me and moves us towards a nice set of "convert arrow --> variant" and back again

cc @carpecodeum @mprammer and @scovich


/// Convert the value at a specific index in the given array into a `Variant`.
#[macro_export]
macro_rules! primitive_conversion_single_value {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

I was thinking about if there is some way to avoid defining 2 macros for each type of array, but given the branches are different (array.is_null --> null or Variant::Null ) I suspect the complexity of refactoring required would outweight any "don't repeat yourself" benefits

@@ -0,0 +1,166 @@
// Licensed to the Apache Software Foundation (ASF) under one
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@alamb
Copy link
Contributor

alamb commented Aug 20, 2025

This PR has a conflict. Can you resolve it @klion26 ?

Thank you!

@alamb
Copy link
Contributor

alamb commented Aug 20, 2025

FYI @scovich

Copy link
Contributor

@scovich scovich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The approach generally looks good, but I'm not sure how it fits with e.g.

(I'm guessing the infrastructure for casting arrays remains useful, and the call/use site would just change if/when the other PR merges?)

Comment on lines 29 to 31
continue;
}
$builder.append_variant(Variant::from(array.value(i)));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: why not just:

Suggested change
continue;
}
$builder.append_variant(Variant::from(array.value(i)));
} else {
$builder.append_variant(Variant::from(array.value(i)));
}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fine with the ruturn/break/continue as soon as possible way, but if we have a strong preference here, I can change it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitely not a strong preference, no need to change if you like it as-is.

(I personally prefer early return/continue/etc any time the else block would be more than one line because it starts to save a lot of indentation. For one line, it's about the same either way)

Comment on lines 85 to 87
macro_rules! non_generic_conversion_array {
($method:ident, $cast_fn:expr, $input:expr, $builder:expr) => {{
let array = $input.$method();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like this is the only "real" macro, and the other two are just special cases?
But for that to work, this macros shouldn't take $input and $method as separate arguments, so caller is e.g.

- non_generic_conversion_array!(as_boolean, |v| v, input, builder);
+ non_generic_conversion_array!(input.as_boolean(), |v| v, builder);

If that's acceptable then we can do:

macro_rules! non_generic_conversion_array {
    ($array:expr, $cast_fn:expr, $builder:expr) => {{
        let array = $array;
          ...
    }};
}

and then

macro_rules! generic_conversion_array {
    ($t:ty, $method:ident, $cast_fn:expr, $input:expr, $builder:expr) => {
        non_generic_conversion_array!($input.$method::<$t>, $cast_fn, $builder)
    };
}

and then

macro_rules! primitive_conversion {
    ($t:ty, $input:expr, $builder:expr) => {
        generic_conversion_array!($t, as_primitive, |v| v, $input, $builder)
    };
}

A similar factoring should work for the xxx_conversion_scalar macros.

Copy link
Member Author

@klion26 klion26 Aug 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added in a separate commit. The code looks cleaner than before. In the new version, there's something changed for the user. If we want to use the primitive_conversion macro, we may need to import the three macros(encounter this in parquet-variant-compute/src/variant_get/output.variant.rs).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh! I forgot about that. You just have to use fully qualified paths, starting with $crate:

macro_rules! primitive_conversion {
    ($t:ty, $input:expr, $builder:expr) => {
        $crate::type_conversion::generic_conversion_array!($t, as_primitive, |v| v, $input, $builder)
    };
}

etc

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And if you want the macro to be usable outside the current crate, then instead:

use self as parquet_variant_compute;

macro_rules! primitive_conversion {
    ($t:ty, $input:expr, $builder:expr) => {
        parquet_variant_compute::type_conversion::generic_conversion_array!($t, as_primitive, |v| v, $input, $builder)
    };
}

... which allows the same path to work both inside and outside the crate.

Tho I suspect it would still break if somebody brought in the crate with a different name?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure those definitions should even be public tho? Why not ditch the #[macro_export] and:

macro_rules! primitive_conversion {
    ($t:ty, $input:expr, $builder:expr) => {
        $crate::type_conversion::generic_conversion_array!($t, as_primitive, |v| v, $input, $builder)
    };
}
pub(crate) use primitive_conversion;

macro_rules! decimal_to_variant_decimal {
($v:ident, $scale:expr, $value_type:ty, $variant_type:ty) => {
if *$scale < 0 {
// For negative scale, we need to multiply the value by 10^|scale|
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

variant doesn't support negative scale?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I see -- you're manually applying the scale and producing a scale=0 decimal result.

($v:ident, $scale:expr, $value_type:ty, $variant_type:ty) => {
if *$scale < 0 {
// For negative scale, we need to multiply the value by 10^|scale|
// For example: 123 with scale -2 becomes 12300
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// For example: 123 with scale -2 becomes 12300
// For example: 123 with scale -2 becomes 12300 with scale 0

if *$scale < 0 {
// For negative scale, we need to multiply the value by 10^|scale|
// For example: 123 with scale -2 becomes 12300
let multiplier = (10 as $value_type).pow((-*$scale) as u32);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
let multiplier = (10 as $value_type).pow((-*$scale) as u32);
let multiplier = <$value_type>::pow(10, (-*$scale) as u32);

Comment on lines 119 to 122
if $v > 0 && $v > <$value_type>::MAX / multiplier {
return Variant::Null;
}
if $v < 0 && $v < <$value_type>::MIN / multiplier {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these fully accurate boundary checks? I worry that the truncating integer division could have corner cases.

I think we can do:

v.checked_mul(multiplier)
    .and_then(|v| <$variant_type>::try_new(v, 0).ok())
    .map_or(Variant::Null, Into::into)

Comment on lines 114 to 132
if *$scale < 0 {
// For negative scale, we need to multiply the value by 10^|scale|
// For example: 123 with scale -2 becomes 12300
let multiplier = (10 as $value_type).pow((-*$scale) as u32);
// Check for overflow
if $v > 0 && $v > <$value_type>::MAX / multiplier {
return Variant::Null;
}
if $v < 0 && $v < <$value_type>::MIN / multiplier {
return Variant::Null;
}
<$variant_type>::try_new($v * multiplier, 0)
.map(|v| v.into())
.unwrap_or(Variant::Null)
} else {
<$variant_type>::try_new($v, *$scale as u8)
.map(|v| v.into())
.unwrap_or(Variant::Null)
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if *$scale < 0 {
// For negative scale, we need to multiply the value by 10^|scale|
// For example: 123 with scale -2 becomes 12300
let multiplier = (10 as $value_type).pow((-*$scale) as u32);
// Check for overflow
if $v > 0 && $v > <$value_type>::MAX / multiplier {
return Variant::Null;
}
if $v < 0 && $v < <$value_type>::MIN / multiplier {
return Variant::Null;
}
<$variant_type>::try_new($v * multiplier, 0)
.map(|v| v.into())
.unwrap_or(Variant::Null)
} else {
<$variant_type>::try_new($v, *$scale as u8)
.map(|v| v.into())
.unwrap_or(Variant::Null)
}
let (v, scale) = if *$scale < 0 {
// For negative scale, we need to multiply the value by 10^-scale
// For example: 123 with scale -2 becomes 12300 with scale 0
let multiplier = <$value_type>::pow(10, (-*$scale) as u32);
($v.checked_mul($v), 0u8)
} else {
(Some($v), *$scale as u8)
};
$v.and_then(|v| <$variant_type>::try_new(v, scale).ok())
.map_or(Variant::Null, Into::into)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed


/// Convert arrays that don't need generic type parameters
#[macro_export]
macro_rules! cast_conversion_nongeneric {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this just non_generic_conversion_array?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I've removed the redundant macros. My bad for not figuring it out when copying it.

Comment on lines 156 to 164
let array = $input.$method::<$offset_type>();
for i in 0..array.len() {
if array.is_null(i) {
$builder.append_null();
continue;
}
let cast_value = $cast_fn(array.value(i));
$builder.append_variant(Variant::from(cast_value));
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Taking the advice above for non_generic_conversion_array:

Suggested change
let array = $input.$method::<$offset_type>();
for i in 0..array.len() {
if array.is_null(i) {
$builder.append_null();
continue;
}
let cast_value = $cast_fn(array.value(i));
$builder.append_variant(Variant::from(cast_value));
}
non_generic_conversion_array!($input.$method::<$offset_type>(), $cast_fn, $builder)

/// Convert the input array of a specific primitive type to a `VariantArray`
/// row by row
#[macro_export]
macro_rules! primitive_conversion {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
macro_rules! primitive_conversion {
macro_rules! primitive_conversion_array {

(to match the others?)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

@klion26 klion26 force-pushed the variant_value_get branch from 783cb22 to f813b08 Compare August 21, 2025 05:52
Copy link
Member Author

@klion26 klion26 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@scovich Thanks for your review, I've updated the code. Please take another look

/// Convert the input array of a specific primitive type to a `VariantArray`
/// row by row
#[macro_export]
macro_rules! primitive_conversion {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

Comment on lines 29 to 31
continue;
}
$builder.append_variant(Variant::from(array.value(i)));
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fine with the ruturn/break/continue as soon as possible way, but if we have a strong preference here, I can change it.


/// Convert arrays that don't need generic type parameters
#[macro_export]
macro_rules! cast_conversion_nongeneric {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I've removed the redundant macros. My bad for not figuring it out when copying it.

Comment on lines 114 to 132
if *$scale < 0 {
// For negative scale, we need to multiply the value by 10^|scale|
// For example: 123 with scale -2 becomes 12300
let multiplier = (10 as $value_type).pow((-*$scale) as u32);
// Check for overflow
if $v > 0 && $v > <$value_type>::MAX / multiplier {
return Variant::Null;
}
if $v < 0 && $v < <$value_type>::MIN / multiplier {
return Variant::Null;
}
<$variant_type>::try_new($v * multiplier, 0)
.map(|v| v.into())
.unwrap_or(Variant::Null)
} else {
<$variant_type>::try_new($v, *$scale as u8)
.map(|v| v.into())
.unwrap_or(Variant::Null)
}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

Comment on lines 85 to 87
macro_rules! non_generic_conversion_array {
($method:ident, $cast_fn:expr, $input:expr, $builder:expr) => {{
let array = $input.$method();
Copy link
Member Author

@klion26 klion26 Aug 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added in a separate commit. The code looks cleaner than before. In the new version, there's something changed for the user. If we want to use the primitive_conversion macro, we may need to import the three macros(encounter this in parquet-variant-compute/src/variant_get/output.variant.rs).

Copy link
Contributor

@scovich scovich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, but please double check that return in primitive_conversion_single_value -- I'm pretty sure it's incorrect and only works by accident today.

generic_conversion_array!(
Float16Type,
as_primitive,
|v: f16| -> f32 { v.into() },
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

aside:

Suggested change
|v: f16| -> f32 { v.into() },
f32::from,

generic_conversion_array!(
Decimal256Type,
as_primitive,
|v: i256| {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

aside: L237-241 below could simplify to just:

                    v.to_i128().map_or(
                        Variant::Null, 
                        decimal_to_variant_decimal!(v, scale, i128, VariantDecimal16,
                    )

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can't change this, because compile thinks that v passed into the macro is i256, and seems we can't cast it when calling the macro.

Copy link
Contributor

@scovich scovich Aug 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I had a silly typo, sorry --

                    v.to_i128().map_or(
                        Variant::Null, 
                        |v| decimal_to_variant_decimal!(v, scale, i128, VariantDecimal16),
                    )

(missing |v| in the closure)

generic_conversion_array!(
Time64NanosecondType,
as_primitive,
|v| NaiveTime::from_num_seconds_from_midnight_opt(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

aside: @alamb -- I'm not sure CI is running fmt against this file? At least, I've never seen it willing to omit trailing commas for non-macro invocations (L412), and it always formats multi-line lambdas with curly braces even tho I'd personally prefer it didn't:

|v| {
    NaiveTime::foo(
        a,
        b,
    )
}

($t:ty, $input:expr, $index:expr) => {{
let array = $input.as_primitive::<$t>();
if array.is_null($index) {
return Variant::Null;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

return from a macro seems dangerous/wrong? It would return from whatever function invoked the macro which is probably not what the caller expected? Is it even what the macro writer intended? To return Variant::Null from the function on NULL, but the macro invocation produces a normal Variant otherwise?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, return from a macro seems wrong, changed the implementation.

@klion26
Copy link
Member Author

klion26 commented Aug 21, 2025

@scovich Thanks for the quick review. I've addressed the comments. Please take another look.

@klion26
Copy link
Member Author

klion26 commented Aug 21, 2025

Fixed in the latest commit
The last commit added #[allow(unused_imports)] before the use half::f16;, because if we don't add this, CI will complain that unused import: half::f16`, but if we remove this import, then the ci complains as this (sample as below)

And also not sure why the CI didn't fail before, the import half::f16 has been added some time ago in #8073.

...
error[E0658]: the type `f16` is unstable
   --> parquet-variant-compute/src/cast_to_variant.rs:827:22
    |
827 |                 Some(f16::MIN),
    |                      ^^^
...

/// Convert the input array to a `VariantArray` row by row, using `method`
/// requiring a generic type to downcast the generic array to a specific
/// array type and `cast_fn` to transform each element to a type compatible with Variant
#[macro_export]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: There's no point (publicly) exporting a macro that references $crate -- compilation will fail for any use sites outside this crate. Can do pub(crate) use generic_conversion_array to make it visible everywhere inside the crate.

@scovich
Copy link
Contributor

scovich commented Aug 21, 2025

The last commit added #[allow(unused_imports)] before the use half::f16;, because if we don't add this, CI will complain that unused import: half::f16`, but if we remove this import, then the ci complains as this (sample as below)

And also not sure why the CI didn't fail before, the import half::f16 has been added some time ago in #8073.

...
error[E0658]: the type `f16` is unstable
   --> parquet-variant-compute/src/cast_to_variant.rs:827:22
    |
827 |                 Some(f16::MIN),
    |                      ^^^
...

Ah! The latter is in the #[cfg(test)] mod test which has a use super::*. When compiling the normal code, the test flag is not set and so the import is indeed unused. If you just move the use half::f16 to mod test the problem should go away.

Comment on lines 66 to 68
macro_rules! generic_conversion_single_value {
($t:ty, $method:ident, $cast_fn:expr, $input:expr, $index:expr) => {{
let array = $input.$method::<$t>();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this not be expressed in terms of non_generic_conversion_single_value?

Comment on lines 89 to 91
macro_rules! primitive_conversion_single_value {
($t:ty, $input:expr, $index:expr) => {{
let array = $input.as_primitive::<$t>();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this not be expressed in terms of generic_conversion_single_value?

@scovich
Copy link
Contributor

scovich commented Aug 21, 2025

Nothing but nits left at this point, can fix them before or after merge.

@klion26
Copy link
Member Author

klion26 commented Aug 21, 2025

@scovich Thank you very much for the detailed and patient review. Addressed the comments in the last commit. One more thing: I changed all the macros to pub(crate) as currently, they are being used in the current compute crate; I think aligning them to the same visibility is better.

@alamb alamb merged commit 81867eb into apache:main Aug 23, 2025
12 checks passed
@alamb
Copy link
Contributor

alamb commented Aug 23, 2025

Thanks again @klion26 and @scovich

@klion26
Copy link
Member Author

klion26 commented Aug 26, 2025

Thanks again very much for the careful review! @alamb @scovich

@klion26 klion26 deleted the variant_value_get branch August 26, 2025 05:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
parquet-variant parquet-variant* crates
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Variant] Implement VariantArray::value for shredded variants
3 participants