Skip to content

feat!: ComposablePass trait allowing sequencing and validation #1895

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 39 commits into from
Apr 22, 2025

Conversation

acl-cqc
Copy link
Contributor

@acl-cqc acl-cqc commented Jan 28, 2025

Currently We have several "passes": monomorphization, dead function removal, constant folding. Each has its own code to allow setting a validation level (before and after that pass).

This PR adds the ability chain (sequence) passes;, and to add validation before+after any pass or sequence; and commons up validation code. The top-level constant_fold_pass (etc.) functions are left as wrappers that do a single pass with validation only in test.

I've left ConstFoldPass as always including DCE, but an alternative could be to return a sequence of the two - ATM that means a tuple (ConstFoldPass, DeadCodeElimPass).

I also wondered about including a method add_entry_point in ComposablePass (e.g. for ConstFoldPass, that means with_inputs but no inputs, i.e. all Top). I feel this is not applicable to all passes, but near enough. This could be done in a later PR but add_entry_point would need a no-op default for that to be a non-breaking change. So if we wouldn't be happy with the no-op default then I could just add it here...

Finally...docs are extremely minimal ATM (this is hugr-passes), I am hoping that most of this is reasonably obvious (it doesn't really do a lot!), but please flag anything you think is particularly in need of a doc comment!

BREAKING CHANGE: quite a lot of calls to current pass routines will break, specific cases include (a) with_validation_level should be done by wrapping a ValidatingPass around the receiver; (b) XXXPass::run() requires use ...ComposablePass (however, such calls will cease to do any validation).

closes #1832

@hugrbot
Copy link
Collaborator

hugrbot commented Jan 28, 2025

This PR contains breaking changes to the public Rust API.

cargo-semver-checks summary

--- failure enum_missing: pub enum removed or renamed ---

Description:
A publicly-visible enum cannot be imported by its prior path. A `pub use` may have been removed, or the enum itself may have been renamed or removed entirely.
      ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
     impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.40.0/src/lints/enum_missing.ron

Failed in:
enum hugr_passes::validation::ValidationLevel, previously in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-passes/src/validation.rs:16
enum hugr_passes::MonomorphizeError, previously in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-passes/src/monomorphize.rs:282
enum hugr_passes::validation::ValidatePassError, previously in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-passes/src/validation.rs:29

--- failure enum_variant_missing: pub enum variant removed or renamed ---

Description:
A publicly-visible enum has at least one variant that is no longer available under its prior name. It may have been renamed or removed entirely.
      ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
     impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.40.0/src/lints/enum_variant_missing.ron

Failed in:
variant ConstFoldError::ValidationError, previously in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-passes/src/const_fold.rs:43
variant ReplaceTypesError::ValidationError, previously in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-passes/src/replace_types.rs:187
variant UntupleError::ValidationError, previously in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-passes/src/untuple.rs:60
variant RemoveDeadFuncsError::ValidationError, previously in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-passes/src/dead_funcs.rs:31

--- failure inherent_method_missing: pub method removed or renamed ---

Description:
A publicly-visible method or associated fn is no longer available under its prior name. It may have been renamed or removed entirely.
      ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
     impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.40.0/src/lints/inherent_method_missing.ron

Failed in:
ConstantFoldPass::validation_level, previously in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-passes/src/const_fold.rs:53
RemoveDeadFuncsPass::validation_level, previously in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-passes/src/dead_funcs.rs:73
MonomorphizePass::validation_level, previously in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-passes/src/monomorphize.rs:290
UntuplePass::validation_level, previously in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-passes/src/untuple.rs:82
UntuplePass::validation_level, previously in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-passes/src/untuple.rs:82
ReplaceTypes::validation_level, previously in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-passes/src/replace_types.rs:211
ReplaceTypes::validation_level, previously in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-passes/src/replace_types.rs:211
DeadCodeElimPass::validation_level, previously in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-passes/src/dead_code.rs:91
DeadCodeElimPass::validation_level, previously in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-passes/src/dead_code.rs:91

--- failure method_parameter_count_changed: pub method parameter count changed ---

Description:
A publicly-visible method now takes a different number of parameters.
      ref: https://doc.rust-lang.org/cargo/reference/semver.html#fn-change-arity
     impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.40.0/src/lints/method_parameter_count_changed.ron

Failed in:
hugr_passes::untuple::UntuplePass::new now takes 1 parameters instead of 2, in /home/runner/work/hugr/hugr/PR_BRANCH/hugr-passes/src/untuple.rs:72
hugr_passes::untuple::UntuplePass::run now takes 2 parameters instead of 3, in /home/runner/work/hugr/hugr/PR_BRANCH/hugr-passes/src/untuple.rs:129
hugr_passes::UntuplePass::new now takes 1 parameters instead of 2, in /home/runner/work/hugr/hugr/PR_BRANCH/hugr-passes/src/untuple.rs:72
hugr_passes::UntuplePass::run now takes 2 parameters instead of 3, in /home/runner/work/hugr/hugr/PR_BRANCH/hugr-passes/src/untuple.rs:129

--- failure method_requires_different_generic_type_params: method now requires a different number of generic type parameters ---

Description:
A method now requires a different number of generic type parameters than it used to. Uses of this method that supplied the previous number of generic types will be broken.
      ref: https://doc.rust-lang.org/reference/items/generics.html
     impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.40.0/src/lints/method_requires_different_generic_type_params.ron

Failed in:
hugr_passes::const_fold::ConstantFoldPass::run takes 0 generic types instead of 1, in /home/runner/work/hugr/hugr/PR_BRANCH/hugr-passes/src/const_fold.rs:91
hugr_passes::RemoveDeadFuncsPass::run takes 0 generic types instead of 1, in /home/runner/work/hugr/hugr/PR_BRANCH/hugr-passes/src/dead_funcs.rs:88
hugr_passes::MonomorphizePass::run takes 0 generic types instead of 1, in /home/runner/work/hugr/hugr/PR_BRANCH/hugr-passes/src/monomorphize.rs:264
hugr_passes::untuple::UntuplePass::run takes 0 generic types instead of 1, in /home/runner/work/hugr/hugr/PR_BRANCH/hugr-passes/src/untuple.rs:129
hugr_passes::UntuplePass::run takes 0 generic types instead of 1, in /home/runner/work/hugr/hugr/PR_BRANCH/hugr-passes/src/untuple.rs:129
hugr_passes::replace_types::ReplaceTypes::run takes 0 generic types instead of 1, in /home/runner/work/hugr/hugr/PR_BRANCH/hugr-passes/src/replace_types.rs:444
hugr_passes::ReplaceTypes::run takes 0 generic types instead of 1, in /home/runner/work/hugr/hugr/PR_BRANCH/hugr-passes/src/replace_types.rs:444

--- failure module_missing: pub module removed or renamed ---

Description:
A publicly-visible module cannot be imported by its prior path. A `pub use` may have been removed, or the module may have been renamed, removed, or made non-public.
      ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
     impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.40.0/src/lints/module_missing.ron

Failed in:
mod hugr_passes::validation, previously in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-passes/src/validation.rs:1

Copy link

codecov bot commented Jan 29, 2025

Codecov Report

Attention: Patch coverage is 89.45455% with 29 lines in your changes missing coverage. Please review.

Project coverage is 83.26%. Comparing base (195f30c) to head (a597008).
Report is 3 commits behind head on release-rs-v0.16.0.

Files with missing lines Patch % Lines
hugr-passes/src/composable.rs 88.77% 17 Missing and 4 partials ⚠️
hugr-passes/src/monomorphize.rs 77.77% 1 Missing and 3 partials ⚠️
hugr-passes/src/replace_types.rs 90.47% 1 Missing and 1 partial ⚠️
hugr-passes/src/const_fold.rs 75.00% 0 Missing and 1 partial ⚠️
hugr-passes/src/dead_funcs.rs 95.65% 1 Missing ⚠️
Additional details and impacted files
@@                  Coverage Diff                   @@
##           release-rs-v0.16.0    #1895      +/-   ##
======================================================
+ Coverage               83.15%   83.26%   +0.10%     
======================================================
  Files                     218      218              
  Lines                   41923    41948      +25     
  Branches                38101    38153      +52     
======================================================
+ Hits                    34861    34926      +65     
+ Misses                   5257     5217      -40     
  Partials                 1805     1805              
Flag Coverage Δ
python 85.42% <ø> (+0.02%) ⬆️
rust 83.04% <89.45%> (+0.11%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.


fn sequence(
self,
other: impl ComposablePass<Err = Self::Err>,
Copy link
Contributor Author

@acl-cqc acl-cqc Mar 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The other way to do this is

fn sequence_before<P2: ComposablePass>(self, other: P2) -> impl ComposablePass<Err = Self::Err>
   where P2::Err : Into<Self::Err>

(and similarly sequence_after) which avoids what may be a common-case use of map_err (i.e. with Into).

However, Infallible doesn't implement Into, so it's probably not much of a win. (I could add sequence_infallible(self, other: ComposablePass<Err=Infallible>) I guess, but would then want that before/after versions of that too)

@acl-cqc acl-cqc marked this pull request as ready for review March 14, 2025 14:40
@acl-cqc acl-cqc requested a review from a team as a code owner March 14, 2025 14:40
@acl-cqc acl-cqc requested a review from cqc-alec March 14, 2025 14:40
Copy link
Collaborator

@cqc-alec cqc-alec left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! I do think some more docs would be good. Regarding the suggestion to add add_entry_point -- does it mean to apply the pass to everything below that point in the hierarchy but not above it? I think that sounds reasonable.

@acl-cqc
Copy link
Contributor Author

acl-cqc commented Apr 15, 2025

The tuple-targets for ComposablePass carry the risk of overcomplicating the API. This is something we expect users to call regularly, so adding arbitrary parameter tupleing may feel unexpected.

@aborgna-q do you mean impl ComposablePass for (A,B)? I have removed these now in favour of a hidden struct as that let me do some magic with error traits and phantomdata.

@acl-cqc
Copy link
Contributor Author

acl-cqc commented Apr 15, 2025

@doug-q @aborgna-q @cqc-alec this has changed quite a bit since Alec approved - as new passes have been added to main, requiring generalization here to handle passes with results. Could (any) one of you please re-review? I've also added a trait for combining two errors (allows combining two methods), and added an "IfThen" pass which needs the first result to return a bool.

I contemplate adding to the trait

fn map<T>(x: impl Fn(Self::Result) -> T) -> impl ComposablePass<Result=T, Error=Self::Error> {...} // impl provided

/// Just like [Self::then] but allows errors to be combined the other way around
fn then2<R2, E: CombineErrors<R2,Self::Error>>(other: ComposablePass<Result=R2, Error=E>) -> impl ComposablePass<Result=(Self::Result,R2), Error=E> {...}

But I guess there is no immediate need - these would be non-breaking if we added them later.

@acl-cqc acl-cqc requested a review from cqc-alec April 16, 2025 10:51
@acl-cqc acl-cqc changed the base branch from main to release-rs-v0.16.0 April 16, 2025 10:51

/// Returns a [ComposablePass] that does "`self` then `other`", so long as
/// `other::Err` can be combined with ours.
fn then<P: ComposablePass, E: ErrorCombiner<Self::Error, P::Error>>(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a possible method then2 that requires E: ErrorCombiner<P::Error, Self::Error> i.e. that would allow when our error-type maps into P's (as opposed to then which requires the other way around).

However, both ways also allow returning an Either error and then2 would reverse that too (you probably wouldn't bother using then2 if you wanted an Either but). So then I wonder, should we also reverse the order of results...and the doubt as to what's the right API has lead me to avoid providing then2 entirely. But I could definitely be persuaded....

(Sadly we can't allow ErrorCombiner to work for both <A, B: Into<A>> and <A: Into<B>, B> because that would provide two conflicting impls when A and B are the same type :-( :-( )

}
}

// Note: in the short term two we could wish for more impls:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// Note: in the short term two we could wish for more impls:
// Note: in the short term we could wish for more impls:

Copy link
Collaborator

@aborgna-q aborgna-q left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

fn from_second(b: B) -> Self;
}

impl<A: Error, B: Into<A>> ErrorCombiner<A, B> for A {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It'd be nice to have

impl<E: Error, A: Into<E>, B: Into<E>> ErrorCombiner<A, B> for E

instead, which covers the Either impl.
But that breaks the tests below as it doesn't allow us to have ErrorCombiner<A, Infallible> -.-

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FTR...AFAICS Either does not implement From from either left or right, e.g. I get two errors here (one on each .into()):

fn foo<A,B>(a:A, b:B, which: bool) -> Either<A,B> {
    if which {a.into()} else {b.into()}
}

Comment on lines +79 to +86
// Note: in the short term we could wish for two more impls:
// impl<E:Error> ErrorCombiner<Infallible, E> for E
// impl<E:Error> ErrorCombiner<E, Infallible> for E
// however, these aren't possible as they conflict with
// impl<A, B:Into<A>> ErrorCombiner<A,B> for A
// when A=E=Infallible, boo :-(.
// However this will become possible, indeed automatic, when Infallible is replaced
// by ! (never_type) as (unlike Infallible) ! converts Into anything
Copy link
Collaborator

@aborgna-q aborgna-q Apr 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is covered by impl<A: Error, B: Error> ErrorCombiner<A, B> for Either<A, B>, no?
It is required to compose DeadCodeElimPass in test_then.

@acl-cqc acl-cqc added this pull request to the merge queue Apr 22, 2025
Merged via the queue into release-rs-v0.16.0 with commit 89c2680 Apr 22, 2025
25 checks passed
@acl-cqc acl-cqc deleted the acl/composable_pass branch April 22, 2025 16:22
aborgna-q pushed a commit that referenced this pull request May 7, 2025
Currently We have several "passes": monomorphization, dead function
removal, constant folding. Each has its own code to allow setting a
validation level (before and after that pass).

This PR adds the ability chain (sequence) passes;, and to add validation
before+after any pass or sequence; and commons up validation code. The
top-level `constant_fold_pass` (etc.) functions are left as wrappers
that do a single pass with validation only in test.

I've left ConstFoldPass as always including DCE, but an alternative
could be to return a sequence of the two - ATM that means a tuple
`(ConstFoldPass, DeadCodeElimPass)`.

I also wondered about including a method `add_entry_point` in
ComposablePass (e.g. for ConstFoldPass, that means `with_inputs` but no
inputs, i.e. all Top). I feel this is not applicable to *all* passes,
but near enough. This could be done in a later PR but `add_entry_point`
would need a no-op default for that to be a non-breaking change. So if
we wouldn't be happy with the no-op default then I could just add it
here...

Finally...docs are extremely minimal ATM (this is hugr-passes), I am
hoping that most of this is reasonably obvious (it doesn't really do a
lot!), but please flag anything you think is particularly in need of a
doc comment!

BREAKING CHANGE: quite a lot of calls to current pass routines will
break, specific cases include (a) `with_validation_level` should be done
by wrapping a ValidatingPass around the receiver; (b) XXXPass::run()
requires `use ...ComposablePass` (however, such calls will cease to do
any validation).

closes #1832
This was referenced May 7, 2025
@hugrbot hugrbot mentioned this pull request May 14, 2025
This was referenced May 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking-change Changes that break semver wait to merge
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Better Pass Infrastructure
5 participants