✨ re-implement dplyr::{filter, mutate, arrange} using data parallelism

[jdhoffa]

See this dicussion:
#8

Acceptance criteria:

• Analogue to dplyr::filter() written
• Analogue to dplyr::mutate() written
• Analogue to dplyr::arrange() written
• Back-end to all should likely involve Rust's rayon crate, and follow the guidance outlined in this chapter: https://rust-lang-nursery.github.io/rust-cookbook/concurrency/parallel.html

[jonocarroll]

One complication I'm interested in is how to support data.frame in general - there are some really funky edge cases that should probably be avoided; you can store a matrix in a data.frame column. You can store a list in a tibble column.

The {savvy} docs do mention working with Rust lists then converting back to data.frame once safely back in the world of R. That might have consequences for filtering a data.frame based on a calculation involving just one column. Perhaps a suitable approach would be calculating a boolean vector in Rust and still doing the actual filtering on the R side.

[jdhoffa]

That is a super interesting point. One easy possibility (to begin with anyway) would be to only support more "standard" data-frame col-types, and error gracefully if we are faced with the funky edge cases?

[jdhoffa]

Re: filter, the idea of only calculating the boolean vector in Rust makes a lot of sense.

[jdhoffa]

Somewhat tangentially related, but I wonder if we should try to support larger-than-memory data structures too:
#26

[asbates]

That is a super interesting point. One easy possibility (to begin with anyway) would be to only support more "standard" data-frame col-types, and error gracefully if we are faced with the funky edge cases?

I think this is probably the best approach. We can do the column type check in R and just never let weird columns enter Rust.