Apache Arrow 5.0: Simplified interaction with the Flight API in Rust

Apache Arrow including the corresponding Rust implementation has been released with numerous bug fixes and new functions in version 5.0. According to their own statements, the development team has fixed a total of 555 problems since Release 4.0, which was presented in May 2021. A large part of this concerns the version of the framework for in-memory data analysis, which is designed for languages ​​such as C, C ++, C #, Go, Java, JavaScript, Julia, MATLAB, Python, R and Ruby, but Rust developers also need to know Set to breaking changes during update.

In Arrow Rust 5.0, the development team tried, among other things, to make the Arrow Flight API easier to use. In order to make the interaction with Flight just as comfortable for Rust developers as they are used to from Python or comparable languages, the conversion measures include constructors and support for useful features of important types. In addition, according to the release announcement, Rust users should also benefit from improved user-friendliness when creating and editing RecordBatchbenefit from it.

A new addition to the Arrow Rust Kernel is the ability to lexicographically partition points and areas so that “Sort by” and “Partition by” are also possible in window functions. A complete compilation of all bug fixes and changes in the Rust implementation summarizes the changelog in the GitHub repository.

The overview of all new features in Arrow 5.0.0 provide the general release notes. While the development team continues to promise new major versions every quarter, the work on the Rust implementation with more frequent minor releases should deliver new functions and bug fixes even faster than before. Meanwhile, the one building on Arrow In-memory query engine DataFusion as well as the Ballista platform for distributed computing decoupled from the Arrow releases and outsourced to their own repositories.


To home page

Leave a Reply

Your email address will not be published. Required fields are marked *