awswrangler conversion

[Thomas-Hirsch]

With pydbtools 3.0.0, the existing functions used by dbtools are being deprecated. While it's possible to run the awswrangler read_sql_query code, the output is a bit weird. Rather than looking like a Pandas dataframe, it seems to look like an R dataframe with Pandas datatypes. This means that reticulate::py_to_r() doesn't work as "it's not a python object", but something like as.tibble doesn't work either as it thinks the columns aren't vectors. Additionally, dbtools currently uses s3tools::get_credentials(). Neither that function nor s3tools in general are necessary any more.

Hence there are two options: either

• to have some custom conversion function to fix the awswrangler output, or
• switch to something more natively R, such as Noctua

[pjrh-moj]

Hi @Thomas-Hirsch - I've just come across this now and can see you've been thinking about a replacement for dbtools in R. As you know, I'm one of the testers of the new R v4 on the AP and so have a particular interest in this - especially since renv doesn't play nicely with Python dependencies at the moment. So I had a go at using noctua to make a version of dbtools which works in R with no Python:
https://github.com/moj-analytical-services/Rdbtools

The noctua package is super easy to use as long as you know the initial parameters to set. I also used this as a chance to learn about S4 classes in R, so the package might be a bit overkill but I learned a lot. Anyway, it's a starter for 10 which I thought I'd let you know about in any case to avoid duplicating too much work. Happy to discuss/adapt/collaborate etc.

Also, Mike Ratford is playing with paws as a replacement for s3tools too (again eliminating the Python dependency).

Thanks,
Peter