Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable Jupyter Book trigger notebook execution functionality #52

Open
g4brielvs opened this issue Jun 11, 2024 · 4 comments
Open

Enable Jupyter Book trigger notebook execution functionality #52

g4brielvs opened this issue Jun 11, 2024 · 4 comments
Assignees
Labels
enhancement New feature or request question Further information is requested

Comments

@g4brielvs
Copy link
Contributor

g4brielvs commented Jun 11, 2024

In light of a recent use case where automated notebook execution could significantly streamline workflows, we propose enabling the functionality to trigger notebook executions selectively. Specifically, we suggest implementing this feature for notebooks located in the reports/ directory while ensuring that notebooks in the notebooks/ directory remain unaffected (execution set to off).

@g4brielvs
Copy link
Contributor Author

@andresfchamorro @bennyistanto @elbeejay @SahitiSarva @avsolatorio @Holly-Transport If you could take a moment to review and share your thoughts, it would be much appreciated. Feel free to comment directly on the issue thread with your insights.

@elbeejay
Copy link
Contributor

It seems like a good idea to me @g4brielvs, I believe documentation is better off when the notebooks are executed, and I think it's important that empty notebooks are committed to the repository. Committing executed notebooks can lead to large "false" changes in PRs due to changes in notebook metadata, as well as merge conflicts, such as in this PR in the GOSTUrban project: worldbank/GOSTurban#32.

In the GOSTUrban project, I raised the issue of using a pre-commit hook to automatically clear notebooks of both their executed content and metadata (worldbank/GOSTurban#24) and have the pre-commit hook, nbstripout, set up in this PR: worldbank/GOSTurban#21

I think it'd be good to make the configuration changes proposed here to execute the notebooks in the "reports" subdirectory via CI when the book is built and add the pre-commit hook nbstripout to the template (involves adding the dependency to the pyproject.toml file and modifying the .pre-commit-config.yaml file to use the hook).

@bennyistanto
Copy link
Member

For me as a viewer, always nice to see the notebook collection from DDP showcase, I am able to play around.

But that's not always worked/useful in my case (notebook related to the economic monitor analysis), from Syria, Turkey, Lebanon, Morocco and Myanmar agriculture monitoring. As most of the process are done locally and required lot of data transfer.

Maybe we can have two options, executed by default (DDP showcase) and not executed for a notebook which the objective is for documentation purpose and not for a showcase (GOST PublicGoods).

@andresfchamorro
Copy link

I agree with the suggestion. This has always been the intention as it makes the work truly reproducible, but there is a bit more complexity managing dependencies to ensure the notebooks run smoothly. I would say let's add some guidelines to readme
Dependencies section explaining how to list packages for executing notebooks etc.

Also agree with Benny that we want to keep the option for notebooks to not be executed. Can think of many cases (some notebooks end up being more experimental or require external platforms/data).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request question Further information is requested
Development

No branches or pull requests

4 participants