Skip to content

chore: add a test case for variable packed struct #5384

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Xuanwo merged 3 commits into from
Dec 5, 2025
Merged

chore: add a test case for variable packed struct #5384

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
e382e83
chore: add a test case for variable packed struct
Xuanwo Dec 2, 2025
9dbe345
Merge branch 'main' into xuanwo/fix-packed-struct
Xuanwo Dec 2, 2025
2c9c4e3
Polish tests
Xuanwo Dec 5, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 47 additions & 0 deletions rust/lance-encoding/src/encodings/physical/packed.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -735,13 +735,17 @@ mod tests {
use crate::{
compression::CompressionStrategy,
compression::{DefaultCompressionStrategy, DefaultDecompressionStrategy},
constants::PACKED_STRUCT_META_KEY,
statistics::ComputeStat,
testing::{check_round_trip_encoding_of_data, TestCases},
version::LanceFileVersion,
};
use arrow_array::{
Array, ArrayRef, BinaryArray, Int32Array, Int64Array, LargeStringArray, StringArray,
StructArray, UInt32Array,
};
use arrow_schema::{DataType, Field as ArrowField, Fields};
use std::collections::HashMap;
use std::sync::Arc;

fn fixed_block_from_array(array: Int64Array) -> FixedWidthDataBlock {
Expand Down Expand Up @@ -947,6 +951,49 @@ mod tests {
Ok(())
}

#[tokio::test]
async fn variable_packed_struct_utf8_round_trip() {
// schema: Struct<id: UInt32, uri: Utf8, long_text: LargeUtf8>
let fields = Fields::from(vec![
Arc::new(ArrowField::new("id", DataType::UInt32, false)),
Arc::new(ArrowField::new("uri", DataType::Utf8, false)),
Copy link
Contributor

@BubbleCal BubbleCal Dec 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would you like to add one more case for LargeUtf8?

Copy link
Collaborator Author

@Xuanwo Xuanwo Dec 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

makes sense

Arc::new(ArrowField::new("long_text", DataType::LargeUtf8, false)),
]);

// mark struct as packed
let mut meta = HashMap::new();
meta.insert(PACKED_STRUCT_META_KEY.to_string(), "true".to_string());

let array = Arc::new(StructArray::from(vec![
(
fields[0].clone(),
Arc::new(UInt32Array::from(vec![1, 2, 3])) as ArrayRef,
),
(
fields[1].clone(),
Arc::new(StringArray::from(vec![
Some("a"),
Some("b"),
Some("/tmp/x"),
])) as ArrayRef,
),
(
fields[2].clone(),
Arc::new(LargeStringArray::from(vec![
Some("alpha"),
Some("a considerably longer payload for testing"),
Some("mid"),
])) as ArrayRef,
),
]));

let test_cases = TestCases::default()
.with_min_file_version(LanceFileVersion::V2_2)
.with_expected_encoding("variable_packed_struct");

check_round_trip_encoding_of_data(vec![array], &test_cases, meta).await;
}

#[test]
fn variable_packed_struct_multi_variable_round_trip() -> Result<()> {
let arrow_fields: Fields = vec![
Expand Down
Loading