Skip to content

Conversation

progval
Copy link

@progval progval commented Jun 9, 2023

This imports tests from https://github.com/apache/orc/tree/main/examples and adds tests which read using this crate, output JSON, and compare with the expected results.

As a side-effect, I also amended write.py to write in that format, and run the same kind of test on those.

This works well for the latter (except test and f32_long_long_gzip because of inconsistent JSON serialization between Python and Rust); but I am struggling to make it work on any test of the official test suite.

Here are the errors I'm getting on the five tests I'm running from the official test suite:

thread 'compare_json::metaData' panicked at 'attempt to subtract with overflow', /home/rust/orc-format/src/read/mod.rs:75:30
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- compare_json::testTimestamp stdout ----
thread 'compare_json::testTimestamp' panicked at 'Timestamp not implemented', tests/it/compare_json.rs:218:22

---- compare_json::test1 stdout ----
thread 'compare_json::test1' panicked at 'Could not deserialize stripe 0 column 2 (boolean1) as kind Byte: OutOfSpec', tests/it/compare_json.rs:60:42

---- compare_json::orc_file_11_format stdout ----
thread 'compare_json::orc_file_11_format' panicked at 'Failed to unwrap metadata: OutOfSpec', tests/it/compare_json.rs:79:49Timestamp not implemented

---- compare_json::testSeek stdout ----
thread 'compare_json::testSeek' panicked at 'index out of bounds: the len is 255 but the index is 255', /home/rust/orc-format/src/read/decode/boolean_rle.rs:119:34

Timestamp not implemented is expected, but I can't make the others work. Any idea if it's a bug in my test code, or limitations in the rest of the crate?

This imports tests from https://github.com/apache/orc/tree/main/examples
and adds tests which read using this crate, output JSON, and compare
with the expected results.

As a side-effect, I also amended write.py to write in that format, and
run the same kind of test on those.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant