Introducing trt_compile() and example how to use it #407
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Introducing lazy TRT compilation support.
Also, a POC of how to add lazy TRT compilation support to MolMIM.
trt_compiler.py as per my plan, will evenually be part of NeMo. (import_utils.py won't be needed for Nemo main as it has safe_import).
Those changes should work across many NeMo versions - I actually had to patch nemo 1.23 to fix export_utils.py bug that was specific to that versions. Most other NeMo versions should work out of the box.
Details
Describe your changes. You can be more detailed and descriptive here.
Usage
How does a user interact with the changed code?
Testing
How do you prove that your code behaves the way you claim?
Tests for these changes can be run via:
(NOTE: also delete this checklist as you fill-out this PR template)
Most of the changes to files with extensions
*.py
,*.yaml
,*.yml
,Dockerfile*
orrequirements.txt
DO REQUIRE bothpytest-
andjet-
CI stages.SKIP_CI
label to your PR?PYTEST_NOT_REQUIRED
label to your PR?JET_NOT_REQUIRED
label to your PR?