Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add async contents manager support #1328

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

Darshan808
Copy link

Fixes #1316

This PR introduces support for an async contents manager. As suggested by @mwouts in #1316, a separate function, build_jupytext_async_contents_manager, has been implemented in a new file, async_contentsmanager.py.

To ensure full test coverage, the existing tests have been duplicated into test_async_contentsmanager.py. Currently, all tests are passing! ✅

Copy link

Thank you for making this pull request.

Did you know? You can try it on Binder: Binder:lab or Binder:notebook.

Also, the version of Jupytext developed in this PR can be installed with pip:

HATCH_BUILD_HOOKS_ENABLE=true pip install git+https://github.com/Darshan808/jupytext.git@async_support

(this requires nodejs, see more at Developing Jupytext)

@Darshan808
Copy link
Author

@mwouts
Could you please review this? I’d appreciate your feedback before proceeding further.

As we begin merging these two files, should we wrap all asynchronous functions (that might be synchronous in the synchronous contents manager) using ensure_async(), similar to what was done in #1021?

Looking forward to your thoughts!

@mwouts
Copy link
Owner

mwouts commented Feb 20, 2025

Fixes #1316

This PR introduces support for an async contents manager. As suggested by @mwouts in #1316, a separate function, build_jupytext_async_contents_manager, has been implemented in a new file, async_contentsmanager.py.

To ensure full test coverage, the existing tests have been duplicated into test_async_contentsmanager.py. Currently, all tests are passing! ✅

Wow, thank you @Darshan808 , this is truly amazing!! Let me have a look!

Copy link
Owner

@mwouts mwouts left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @Darshan808 , I love what I see so far!

As we begin merging these two files, should we wrap all asynchronous functions (that might be synchronous in the synchronous contents manager) using ensure_async(), similar to what was done in #1021?

Do you think it would be possible to do so? It would be great indeed if we had a more local change - easier to review, and just one file to maintain later on. But I must admit that it's precisely what I was not able to do two years ago so no obligation at all!

try:
from jupytext.async_contentsmanager import build_jupytext_async_contents_manager_class
except ImportError as err:
build_jupytext_async_contents_manager = reraise(err)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes that's the most prudent approach, thank you! The import problem was years ago though, so maybe it's gone, but it's safer that I follow-up on that later on.

if asyncio.iscoroutinefunction(base_class.get):
app.contents_manager_class = build_jupytext_async_contents_manager_class(base_class)
else:
app.contents_manager_class = build_jupytext_contents_manager_class(base_class)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes exactly what I had in mind! And we can delete the paragraph above.

@mwouts
Copy link
Owner

mwouts commented Feb 20, 2025

Re the failing pre-commit hook on the CI you probably just need pre-commit install - see also https://jupytext.readthedocs.io/en/latest/developing.html#install-and-develop-jupytext-locally

@Darshan808 Darshan808 marked this pull request as ready for review February 21, 2025 16:31
@Darshan808
Copy link
Author

Darshan808 commented Feb 21, 2025

@mwouts
I merged the two files, wrapping methods with ensure_async(), and merged the tests as well. To support both synchronous and asynchronous ContentsManager, I added a pytest fixture that provides both managers to all tests. All tests are passing successfully.
Let me know what you think about this change!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support AsyncContentsManager?
2 participants