Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introducing trt_compile() and example how to use it #407

Open
wants to merge 6 commits into
base: bionemo1
Choose a base branch
from

Conversation

borisfom
Copy link
Collaborator

@borisfom borisfom commented Nov 6, 2024

Summary

Introducing lazy TRT compilation support.
Also, a POC of how to add lazy TRT compilation support to MolMIM.
trt_compiler.py as per my plan, will evenually be part of NeMo. (import_utils.py won't be needed for Nemo main as it has safe_import).
Those changes should work across many NeMo versions - I actually had to patch nemo 1.23 to fix export_utils.py bug that was specific to that versions. Most other NeMo versions should work out of the box.

Details

Describe your changes. You can be more detailed and descriptive here.

Usage

How does a user interact with the changed code?

python -m your.new.module -and -all -options

Testing

How do you prove that your code behaves the way you claim?

Tests for these changes can be run via:

pytest -v tests/your/new/or/existing/test_functions.py::test_function

(NOTE: also delete this checklist as you fill-out this PR template)

Most of the changes to files with extensions *.py, *.yaml, *.yml, Dockerfile* or requirements.txt DO REQUIRE both pytest- and jet- CI stages.

Signed-off-by: Boris Fomitchev <[email protected]>
@borisfom borisfom requested a review from trvachov November 6, 2024 22:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant