Info for chunking to fix #52

[annefou]

I started to draft the chunking section to fix #52.

[annefou]

@tinaok I started to write some information about the chunking. It needs to be refined that is why I opened a draft PR.

[felixcremer]

Feel free to incorporate parts of https://github.com/linamaes/chunking_tutorial if you find it useful.

[annefou]

Feel free to incorporate parts of https://github.com/linamaes/chunking_tutorial if you find it useful.

Great! thank you!

[annefou]

@clausmichele @tinaok what do you think about having an exercise on chunking?

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[tinaok]

@annefou end of the chunking_introduction_snow.ipynb, it says

Computations on big datasets can be very slow on a single computer, and to optimize its time we may need to parallelize your computations. This is what you will learn in the next episode with Dask.

Is this somewhere in this branch or it is in other branch? or it is not yet in this repo?

[annefou]

We are using dask when creating the snow map but we need here to teach them how to use dask. It needs to be added in the same notebook.

[tinaok]

• 2.4_exercises/chunking_introduction_snow.ipynb the setup section's package list, do we keep it? if yes we need to update the package list (for example we do not use matplotlib but use hvplot)

[tinaok]

We are using dask when creating the snow map but we need here to teach them how to use dask. It needs to be added in the same notebook.

I got the scaling_dask notebook, we can update to fit in the story.

[clausmichele]

@tinaok @annefou my review:

• Paths: please use cubes-and-clouds/lectures/2.4_formats_and_performance/assets/ for images and auxiliary files and cubes-and-clouds/lectures/2.4_formats_and_performance/exercises/ for the notebooks.

Lecture (markdown):

• Questions for the quiz are missing, please add a couple of questions.
• Where would you like the user to start your notebooks? The exercises are not mentioned in the lecture.

Notebooks:

• chunking_introduction_snow.ipynb and scaling_dask.ipynb:
  • there are too many things in them, remove what you already describe in the lecture. Good that you want to reuse content which was already developed, but you need to reduce the content, otherwise the users might feel overwhelmed.
  • the layout is not aligned with the other notebooks in the course.

[tinaok]

@clausmichele
other than this part

> ### Cost of scalability
> 
> Direct examples of computing on a workflow
> (todo: based on actual workflow)
> 
> ### Memory consumption
> 
> limitations and 
> 
> ### Difference between platform usage and cloud directly
> 
> TODO: Is this covered already in the platform lesson? Yes
> Using platforms removes complexity and adds abstraction layers

And additional Questions for quiz, it is done.
Can you please review?
thank you.
cc: @annefou