One large package or many small packages for different datasets?

[natgeo-wong]

Need a second opinion ...

Many (i.e. around 3-4+) small packages for dealing with reanalysis datasets (one for each major dataset), like, one for ERA5, one for NCEP, one for MERRA, one for ERA-Interim, etc., or

1 large package allowing one to download/process all the datasets together? Going to be tricky because NCEP's data is arranged quite differently (esp. pressure levels) compared to ECMWF reanalysis datasets.

Same would go for different satellite datasets. Thinking about one for GPM, one for TRMM, and then going for the MODIS, and other different satellites.

[Datseris]

my vote: 1 large.

[Alexander-Barth]

I would also tend to use 1 large package, unless this would lead to a package with a very large number of dependencies (or tricky to install dependencies, like a python module).

[juliohm]

Another vote for 1 large package.

Also appreciate if you can clarify how your idea relates to CDSAPI.jl for example, which is a package that already contains a bunch of climate datasets.

[Balinus]

I'd also prefer 1 package, as this would help people discovering unknown datasets (unkonwn to the user I mean).

[natgeo-wong]

Alright! Sorry for the slow response, trying to catch up with everything and it was holiday season!

Will slowly be working on this in the coming months. Need to pick up where I left off.

[gaelforget]

Just linking #14 to this related discussion thread

[juliohm]

My opinion about this issue changed over time. I think it depends on the scope of the "large package". If there is a common theme like a package to load "raster data", then it may make sense to group data sources. In any case, it is reasonable to start with specific packages that do their job well and then consider later if it makes sense to merge them into a common API.

[natgeo-wong]

Yeah that was my thought. I'm thinking of doing several different satellite dataset packages (mostly those that I might use), and the maybe eventually combine them in ClimateSatellite.jl by calling them and reexporting them.