We created Parquet to make the advantages of compressed, efficient columnar data representation available to any project in the Hadoop ecosystem. Parquet is built from the ground up with complex nested data structures in mind, and uses the record shredding and assembly algorithm described in the Dremel paper. We believe this approach is superior to simple flattening of nested name spaces. Parquet is built to support very efficient compression and encoding schemes.| Apache Parquet
A complete, safe, native Rust implementation of Apache Arrow, a cross-language development platform for in-memory data.| docs.rs
Physical Memory Layout#| arrow.apache.org