Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Column type validation issue on date columns in parquet submissions #118

Closed
ruarai opened this issue Sep 10, 2024 · 1 comment · Fixed by #119
Closed

Column type validation issue on date columns in parquet submissions #118

ruarai opened this issue Sep 10, 2024 · 1 comment · Fixed by #119
Assignees

Comments

@ruarai
Copy link

ruarai commented Sep 10, 2024

Our hub has run into an issue with the column type checking. It seems submitting a parquet file with date-times in a date column will raise an error in check_tbl_col_types.R:

[col_types]: EXEC ERROR: Error in purrr::map_chr(tbl, ~if (inherits(.x,
"numeric")) [ : ℹ In index: 1. ℹ With name: forecast_date. Caused by error: !
Result must be length 1, not 2.

This can be resolved of course by making sure the relevant columns are of type date rather than date-time (but this is not super obvious from this error to our submitters, at least).

The error seems to arise as class(x) for a date-time vector will return a vector of two values, i.e.:

> class(Sys.time())
[1] "POSIXct" "POSIXt" 

I was going to submit a PR for a simple fix to just return the first element from class(.x) - but worried this could cause issues later?

Sorry I don't have a reprex for this one.

@annakrystalli
Copy link
Member

annakrystalli commented Sep 11, 2024

Thanks for the bug report @ruarai !

Now sorted and we'll also be adding a section in our docs regarding this. See hubverse-org/hubDocs#177 and https://github.com/orgs/hubverse-org/discussions/31

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Development

Successfully merging a pull request may close this issue.

2 participants