Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cube of dataset with a 0-dimensional subcube gives confusing error message #218

Open
felixcremer opened this issue Jan 31, 2023 · 2 comments

Comments

@felixcremer
Copy link
Member

I was trying to write up an example for the JuliaClimate meta issue about julia packages for the CMIP6 data
JuliaClimate/meta#17 and I ran into the following error with trying to convert a CMIP6 dataset into a YAXArray Cube.
The problem is, that the dataset I randomly selected seem to have a zero dimensional subcube which can't be used to derive the size of the expected data cube from the concatenation.

julia> using AWS, Zarr, YAXArrays, CSV, DataFrames

julia> AWS.global_aws_config(AWSConfig(creds=nothing, region=""))
AWSConfig(nothing, "", "json")

julia> # table with all the URLs to the data on google cloud
       CMIP6_stores = CSV.read(download("https://storage.googleapis.com/cmip6/cmip6-zarr-consolidated-stores.csv"), DataFrame)

julia> store = filter(CMIP6_stores) do row
         row.activity_id == "ScenarioMIP" && row.institution_id=="DKRZ" && row.variable_id=="tas" && row.experiment_id=="ssp585"
       end
3×11 DataFrame
 Row │ activity_id  institution_id  source_id      experiment_id  member_id  table_id  variable_id  grid_label  zstore                             dcpp_init_year  version  
     │ String15     String31        String31       String31       String15   String15  String31     String7     String                             Float64?        Int64    
─────┼──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1 │ ScenarioMIP  DKRZ            MPI-ESM1-2-HR  ssp585         r1i1p1f1   day       tas          gn          gs://cmip6/CMIP6/ScenarioMIP/DKR         missing  20190710
   2 │ ScenarioMIP  DKRZ            MPI-ESM1-2-HR  ssp585         r1i1p1f1   Amon      tas          gn          gs://cmip6/CMIP6/ScenarioMIP/DKR         missing  20190710
   3 │ ScenarioMIP  DKRZ            MPI-ESM1-2-HR  ssp585         r1i1p1f1   3hr       tas          gn          gs://cmip6/CMIP6/ScenarioMIP/DKR         missing  20190710

julia> zopen(store.

activity_id     dcpp_init_year  experiment_id   grid_label      institution_id  member_id       source_id       table_id        variable_id     version         zstore
julia> zopen(store.zstore[1])
ZarrGroup at GCStore("cmip6") and path CMIP6/ScenarioMIP/DKRZ/MPI-ESM1-2-HR/ssp585/r1i1p1f1/day/tas/gn/v20190710/
Variables: lat height time tas lat_bnds lon_bnds lon time_bnds 

julia> zopen(store.zstore[1])
ZarrGroup at GCStore("cmip6") and path CMIP6/ScenarioMIP/DKRZ/MPI-ESM1-2-HR/ssp585/r1i1p1f1/day/tas/gn/v20190710/
Variables: lat height time tas lat_bnds lon_bnds lon time_bnds 

julia> @time zopen(store.zstore[1])
  1.190920 seconds (6.51 k allocations: 692.555 KiB)
ZarrGroup at GCStore("cmip6") and path CMIP6/ScenarioMIP/DKRZ/MPI-ESM1-2-HR/ssp585/r1i1p1f1/day/tas/gn/v20190710/
Variables: lat height time tas lat_bnds lon_bnds lon time_bnds 

julia> c = Cube(store.zstore[1])
ERROR: ArgumentError: invalid index: nothing of type Nothing
Stacktrace:
  [1] to_index(i::Nothing)
    @ Base ./indices.jl:300
  [2] to_index(A::Vector{Any}, i::Nothing)
    @ Base ./indices.jl:277
  [3] to_indices
    @ ./indices.jl:333 [inlined]
  [4] to_indices
    @ ./indices.jl:325 [inlined]
  [5] getindex
    @ ./abstractarray.jl:1241 [inlined]
  [6] concatenatecubes(cl::Vector{Any}, cataxis::CategoricalAxis{String, :Variable, Vector{String}})
    @ YAXArrays.Cubes ~/.julia/packages/YAXArrays/Fe7F8/src/Cubes/TransformedCubes.jl:31
  [7] Cube(ds::Dataset; joinname::String)
    @ YAXArrays.Datasets ~/.julia/packages/YAXArrays/Fe7F8/src/DatasetAPI/Datasets.jl:324
  [8] Cube
    @ ~/.julia/packages/YAXArrays/Fe7F8/src/DatasetAPI/Datasets.jl:299 [inlined]
  [9] #Cube#122
    @ ~/.julia/packages/YAXArrays/Fe7F8/src/DatasetAPI/Datasets.jl:744 [inlined]
 [10] Cube(s::String)
    @ YAXArrays.Datasets ~/.julia/packages/YAXArrays/Fe7F8/src/DatasetAPI/Datasets.jl:744
 [11] top-level scope
    @ REPL[15]:1
@lazarusA
Copy link
Collaborator

lazarusA commented Jan 31, 2023

ahhh, again this issue. @meggart did fixed it before, but I don't see the commit anywhere now 😢 . Also see #165
Ahh, this one was suppose to fix it #199 , related issue also JuliaDataCubes/EarthDataLab.jl#284

@Balinus
Copy link
Contributor

Balinus commented Feb 23, 2023

Once completed, this example woule be handy in the documentation. Cheers!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants