Added note on available datatypes for DP1 that we will handle #5
Conversation
| Note that more complex types like arrays of numerical types are not on our schedule for the initial release (DP1), but we may decide to | ||
| extend our capabilities for handling these types if it becomes a requirement for future releases. | ||
|
|
||
|
|
There was a problem hiding this comment.
How about timestamps? When I was looking into the Parquet files generated by the pipelines, I found 5 columns where the timestamps were encoded:
- Table
Visithas two columns:expMidptandobsStart - Table
CcdVisithas two columns:expMidptandobsStart - Table
DiaSourcehas one columntime_processed
The Parquet low-level schema for the columns is:
% parq diaSourceTable_tract_10463_lsst_cells_v1_LSSTComCam_runs_DRP_DP1_w_2025_09_DM-49235_20250303T090128Z.parq --schema | grep Timestamp
optional int64 field_id=-1 time_processed (Timestamp(isAdjustedToUTC=false, timeUnit=nanoseconds, is_from_converted_type=false, force_set_converted_type=false));
There was a problem hiding this comment.
Forgot to mention table ObsCore. It has a couple of interesting SQL types:
There was a problem hiding this comment.
I should maybe clarify this in the document, but the types I was referring to are what the IVOA datatypes, i..e what appear in the VOTable & TAP_SCHEMA.
For timestamps for example, we currently output as a char (). So Russ will have to format whatever the encoding is in QServ into a char(). Similar for the Obscore table, we map those complicated SQL types into varchars.
No description provided.