Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ajout d'outils et modèles simples de LSTM #82

Open
wants to merge 53 commits into
base: main
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
6571ad1
Create lstm branch
richardarsenault Feb 3, 2024
f923ed2
Add files via upload
richardarsenault Feb 3, 2024
d9e7171
Add files via upload
richardarsenault Feb 3, 2024
402b1cd
first rough pass
richardarsenault Feb 3, 2024
d21478e
first rough pass
richardarsenault Feb 3, 2024
6e1b77c
setup and cleaning of basic tests
richardarsenault Feb 3, 2024
44316df
continuing cleanup of first draft
richardarsenault Feb 4, 2024
980e478
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Feb 4, 2024
f6acae0
started fixing lint
richardarsenault Feb 4, 2024
acf070e
fixed linting
richardarsenault Feb 4, 2024
5bb9f2b
fixed linting after rebase
richardarsenault Feb 4, 2024
0f8c609
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Feb 4, 2024
79389bb
fixed minor linting issues
richardarsenault Feb 4, 2024
8da2853
Merge branch 'lstm' of github.com:hydrologie/xhydro into lstm
richardarsenault Feb 4, 2024
c642ae4
fixed linting/docs in lstm tests
richardarsenault Feb 4, 2024
8eaf80a
small fix
richardarsenault Feb 4, 2024
199b08e
updating tests with smaller files
richardarsenault Feb 4, 2024
4eed834
Merge remote-tracking branch 'origin/main' into lstm
richardarsenault Feb 10, 2024
160d586
trying to fix lstm package problems
richardarsenault Feb 10, 2024
619c953
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Feb 10, 2024
2df3842
pytest works, not pycharm
richardarsenault Feb 10, 2024
77f9a49
pytest works, not pycharm
richardarsenault Feb 10, 2024
6159aed
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Feb 10, 2024
c584297
Add single basin LSTM models
richardarsenault Feb 11, 2024
5eea1f0
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Feb 11, 2024
0b138df
finished updating LSTM first pass
richardarsenault Feb 11, 2024
b9ec1c0
Merge branch 'lstm' of github.com:hydrologie/xhydro into lstm
richardarsenault Feb 11, 2024
ebc1e0a
fixed import problem
richardarsenault Feb 12, 2024
bb23560
Merge branch 'main' into lstm
Zeitsperre Feb 29, 2024
707f4d9
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Feb 29, 2024
def070c
oops
Zeitsperre Feb 29, 2024
4959b66
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Feb 29, 2024
424d8b0
merged origin into branch, fixed conflicts
richardarsenault Mar 7, 2024
b592be5
fix conda install
Zeitsperre Mar 7, 2024
8c1d4f7
applied changes according to review
richardarsenault Mar 8, 2024
f479f54
Merge branch 'lstm' of github.com:hydrologie/xhydro into lstm
richardarsenault Mar 8, 2024
e94aa8d
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 8, 2024
d3a5ba0
fix some precommit errors, but all 'star imports' errors remain
richardarsenault Mar 8, 2024
9937687
fixed auto merge again
richardarsenault Mar 8, 2024
be9bb36
updated gh path for lstm test data
richardarsenault Mar 8, 2024
281f372
merge conflicts
richardarsenault Mar 22, 2024
a6819a6
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 22, 2024
c9b7035
review changes and pre-commit fixes
richardarsenault Mar 23, 2024
ac87553
Merge branch 'lstm' of github.com:hydrologie/xhydro into lstm
richardarsenault Mar 23, 2024
7f77f4a
Added notebook and added some functionality to better access custom L…
richardarsenault Mar 24, 2024
8953aa5
Update lstm.ipynb
richardarsenault Mar 26, 2024
f3a553a
Merge branch 'main' into lstm
richardarsenault Mar 26, 2024
3dff6ef
Merge branch 'main' into lstm
richardarsenault Apr 12, 2024
728277b
Merge branch 'main' into lstm
Zeitsperre Feb 20, 2025
a699ab8
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Feb 20, 2025
f80bb04
Merge branch 'main' into lstm
Zeitsperre Feb 25, 2025
41c0163
move lstm to new structure, render pytorch/lstm optional, allow makef…
Zeitsperre Feb 25, 2025
3f57b9b
docstring and grammar fixes, permitted naming of submodule
Zeitsperre Feb 25, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fixed minor linting issues
richardarsenault committed Feb 4, 2024

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
commit 79389bbfbcb3768a4b0e79cf21483930f41911f6
5 changes: 3 additions & 2 deletions xhydro/lstm_tools/LSTM_static.py
Original file line number Diff line number Diff line change
@@ -1,9 +1,12 @@
"""LSTM model definition and tools for LSTM model training."""

import math

import numpy as np
import tensorflow as tf
import tensorflow.keras.backend as k
from tensorflow.keras.models import load_model

from .create_datasets import create_lstm_dataset


@@ -49,7 +52,6 @@ def __len__(self):

def __getitem__(self, idx):
"""Get one of the batches by taking the 'batch_size' first elements from the randomized list of remaining indices."""

inds = self.indices[idx * self.batch_size : (idx + 1) * self.batch_size]
batch_x = self.x[inds]
batch_x_static = self.x_static[inds]
@@ -348,7 +350,6 @@ def run_trained_model(
flows : np.array
Observed and simulated streamflows computed for the watershed of interest and for a specified period.
"""

# Delete and reload the model to free the memory
k.clear_session()
model_lstm = load_model(
1 change: 1 addition & 0 deletions xhydro/lstm_tools/create_datasets.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""Tools to create the datasets to be used in LSTM model training and simulation."""

import numpy as np
import xarray as xr

1 change: 1 addition & 0 deletions xhydro/lstm_tools/lstm_controller.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""Control the LSTM training and simulation tools to make clean workflows."""

import os
import tempfile