Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improved docs #46

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 11 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,17 +24,17 @@

### Project description

We plan to investigate various machine learning (ML) techniques for predicting floods. The main goal is a comparative study of some of the most promising ML methods on this proposed subject. As a side goal, the open source development approach via github will result in a nice basis for further work.
We plan to investigate various machine learning (ML) techniques for predicting floods. The main goal is a comparative study of some of the most promising ML methods on this proposed subject. As a side goal, the open-source development approach via GitHub will result in a nice basis for further work.

ERA5 data will be used as predictor to model either the probability of exceeding some threshold in river discharge by the GloFAS reanalysis or to predict the severeness of the event given by the ECMWF’s severe event catalogue. We plan to investigate the impact of different meteorological variables, starting with direct precipitation output and combinations of thermodynamic and dynamic variables.
Additionally, the results can be compared with GloFAS forecast reruns as well. Thereby, the benefits and/or drawbacks of using ML techniques instead of coupled complex models can be explored.
ERA5 data will be used as a predictor to model either the probability of exceeding some threshold in river discharge by the GloFAS reanalysis or to predict the severeness of the event given by the ECMWF’s severe event catalog. We plan to investigate the impact of different meteorological variables, starting with direct precipitation output and combinations of thermodynamic and dynamic variables.
Additionally, the results can be compared with the GloFAS forecast reruns as well. Thereby, the benefits and/or drawbacks of using ML techniques instead of coupled complex models can be explored.

Our projected workflow can be seen below:

<img src="https://raw.githubusercontent.com/esowc/ml_flood/master/notebooks/resources/MATEHIW_flowchart.png" alt="model-steps" width="600"/>

### Dependencies and Setup
**This repository was created for Python3** Dependencies can be found in the **environment.yml** file. Download the repository, move it to any path you wish for. You can either install all packages by hand, or you can use `conda env create -f environment.yml` for a one-step installation of all dependencies. When installed, a new environment named **ml_flood** is created. Remember to use
**This repository was created for Python3** Dependencies can be found in the **environment.yml** file. Download the repository, move it to any path you wish for. You can either install all packages by hand, or you can use `conda env create -f environment.yml` for one-step installation of all dependencies. When installed, a new environment named **ml_flood** is created. Remember to use
`bash; conda activate ml_flood`
before executing any script to ensure all packages exist.
To start `jupyter` in a specific conda environment, we had to activate the ipython kernel with
Expand All @@ -49,8 +49,8 @@ To experiment with the notebooks, download the repository to your local device:
```sh
git clone https://github.com/esowc/ml_flood.git
```
A folder `ml_flood` has been created. It not only includes the notebooks but also a dataset for you to experiment around and develop your own extensions to existing models.
The folder structure will be as you would expect from the github webpage:
A folder `ml_flood` has been created. It not only includes the notebooks but also a dataset for you to experiment around and develop your extensions to existing models.
The folder structure will be as you would expect from the GitHub webpage:
```
.
+-- data/
Expand All @@ -63,7 +63,7 @@ The folder structure will be as you would expect from the github webpage:
+-- python/
| +-- aux/
```
The `data/` folder contains the small test dataset included in the repository. The notebooks folder contains all fully-reproducible notebooks that work with the small test dataset, except for the `4_coupled_model/` folder. The `python/` folder contains work in progress scripts that were written in the process of creating this repo and may contain errors or be incomplete. The `python/aux/` folder contains collections of code that is used throughout the notebooks.
The `data/` folder contains the small test dataset included in the repository. The notebooks folder contains all fully-reproducible notebooks that work with the small test dataset, except for the `4_coupled_model/` folder. The `python/` folder contains work in progress scripts that were written in the process of creating this repo and may contain errors or be incomplete. The `python/aux/` folder contains collections of code that are used throughout the notebooks.


### Data description
Expand All @@ -72,15 +72,14 @@ For reproducibility, a small testing dataset is included in the folder `./data/`

### Model structure
We implemented two major structures of ML models:
- The simpler, catchment based model
which predicts the timeseries of discharge at a certain point given *upstream* precipitation etc. from ERA5.
- The simpler, catchment-based model which predicts the time-series of discharge at a certain point given *upstream* precipitation etc. from ERA5.
- The more complex, regional *coupled model*
which predicts the next state (timestep) of river discharge from previous states (discharges) and water input (from the atmosphere). It is physics inspired and splits up the prediction of discharge at a certain point into
which predicts the next state (timestep) of river discharge from previous states (discharges) and water input (from the atmosphere). It is physics-inspired and splits up the prediction of discharge at a certain point into
- water inflow from upstream river gridpoints and
- water inflow from subsurface flow and smaller rivers.
- water inflow from the subsurface flow and smaller rivers.
The model is fitted for every river gridpoint separately, thus making the training process more complex than applying a single model to all gridpoints.

The model structure of the regional *coupled model* is layed out in the flowchart below. The model takes the influence of different features as well as their spatial and temporal state into account by spliting the whole process up into two models. The first encompasses changes in discharge happening due to non-local reasons (e.g. large-scale precipitation a few hundred kilometres upstream, affecting the flow at a certain point a few days later through river flow) and the second includes local effects from features like precipitation/runoff and their impact on discharge by subsurface flow or smaller rivers. For more detail see the notebooks in the **/notebooks/** folder.
The model structure of the regional *coupled model* is layed out in the flowchart below. The model takes the influence of different features as well as their spatial and temporal state into account by splitting the whole process up into two models. The first encompasses changes in discharge happening due to non-local reasons (e.g. large-scale precipitation a few hundred kilometers upstream, affecting the flow at a certain point a few days later through river flow), and the second includes local effects from features like precipitation/runoff and their impact on discharge by subsurface flow or smaller rivers. For more detail see the notebooks in the **/notebooks/** folder.

<img src="https://raw.githubusercontent.com/esowc/ml_flood/master/notebooks/resources/model-steps_v2-1.png" alt="model-steps" width="500"/>

Expand Down