feat!: ComposablePass trait allowing sequencing and validation #1895

acl-cqc · 2025-01-28T15:40:58Z

Currently We have several "passes": monomorphization, dead function removal, constant folding. Each has its own code to allow setting a validation level (before and after that pass).

This PR adds the ability chain (sequence) passes;, and to add validation before+after any pass or sequence; and commons up validation code. The top-level constant_fold_pass (etc.) functions are left as wrappers that do a single pass with validation only in test.

I've left ConstFoldPass as always including DCE, but an alternative could be to return a sequence of the two - ATM that means a tuple (ConstFoldPass, DeadCodeElimPass).

I also wondered about including a method add_entry_point in ComposablePass (e.g. for ConstFoldPass, that means with_inputs but no inputs, i.e. all Top). I feel this is not applicable to all passes, but near enough. This could be done in a later PR but add_entry_point would need a no-op default for that to be a non-breaking change. So if we wouldn't be happy with the no-op default then I could just add it here...

Finally...docs are extremely minimal ATM (this is hugr-passes), I am hoping that most of this is reasonably obvious (it doesn't really do a lot!), but please flag anything you think is particularly in need of a doc comment!

BREAKING CHANGE: quite a lot of calls to current pass routines will break, specific cases include (a) with_validation_level should be done by wrapping a ValidatingPass around the receiver; (b) XXXPass::run() requires use ...ComposablePass (however, such calls will cease to do any validation).

closes #1832

hugrbot · 2025-01-28T15:45:50Z

This PR contains breaking changes to the public Rust API.

cargo-semver-checks summary


--- failure enum_missing: pub enum removed or renamed ---

Description:
A publicly-visible enum cannot be imported by its prior path. A `pub use` may have been removed, or the enum itself may have been renamed or removed entirely.
      ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
     impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.40.0/src/lints/enum_missing.ron

Failed in:
enum hugr_passes::validation::ValidationLevel, previously in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-passes/src/validation.rs:16
enum hugr_passes::MonomorphizeError, previously in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-passes/src/monomorphize.rs:282
enum hugr_passes::validation::ValidatePassError, previously in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-passes/src/validation.rs:29

--- failure enum_variant_missing: pub enum variant removed or renamed ---

Description:
A publicly-visible enum has at least one variant that is no longer available under its prior name. It may have been renamed or removed entirely.
      ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
     impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.40.0/src/lints/enum_variant_missing.ron

Failed in:
variant ConstFoldError::ValidationError, previously in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-passes/src/const_fold.rs:43
variant ReplaceTypesError::ValidationError, previously in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-passes/src/replace_types.rs:187
variant UntupleError::ValidationError, previously in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-passes/src/untuple.rs:60
variant RemoveDeadFuncsError::ValidationError, previously in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-passes/src/dead_funcs.rs:31

--- failure inherent_method_missing: pub method removed or renamed ---

Description:
A publicly-visible method or associated fn is no longer available under its prior name. It may have been renamed or removed entirely.
      ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
     impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.40.0/src/lints/inherent_method_missing.ron

Failed in:
ConstantFoldPass::validation_level, previously in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-passes/src/const_fold.rs:53
RemoveDeadFuncsPass::validation_level, previously in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-passes/src/dead_funcs.rs:73
MonomorphizePass::validation_level, previously in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-passes/src/monomorphize.rs:290
UntuplePass::validation_level, previously in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-passes/src/untuple.rs:82
UntuplePass::validation_level, previously in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-passes/src/untuple.rs:82
ReplaceTypes::validation_level, previously in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-passes/src/replace_types.rs:211
ReplaceTypes::validation_level, previously in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-passes/src/replace_types.rs:211
DeadCodeElimPass::validation_level, previously in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-passes/src/dead_code.rs:91
DeadCodeElimPass::validation_level, previously in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-passes/src/dead_code.rs:91

--- failure method_parameter_count_changed: pub method parameter count changed ---

Description:
A publicly-visible method now takes a different number of parameters.
      ref: https://doc.rust-lang.org/cargo/reference/semver.html#fn-change-arity
     impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.40.0/src/lints/method_parameter_count_changed.ron

Failed in:
hugr_passes::untuple::UntuplePass::new now takes 1 parameters instead of 2, in /home/runner/work/hugr/hugr/PR_BRANCH/hugr-passes/src/untuple.rs:72
hugr_passes::untuple::UntuplePass::run now takes 2 parameters instead of 3, in /home/runner/work/hugr/hugr/PR_BRANCH/hugr-passes/src/untuple.rs:129
hugr_passes::UntuplePass::new now takes 1 parameters instead of 2, in /home/runner/work/hugr/hugr/PR_BRANCH/hugr-passes/src/untuple.rs:72
hugr_passes::UntuplePass::run now takes 2 parameters instead of 3, in /home/runner/work/hugr/hugr/PR_BRANCH/hugr-passes/src/untuple.rs:129

--- failure method_requires_different_generic_type_params: method now requires a different number of generic type parameters ---

Description:
A method now requires a different number of generic type parameters than it used to. Uses of this method that supplied the previous number of generic types will be broken.
      ref: https://doc.rust-lang.org/reference/items/generics.html
     impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.40.0/src/lints/method_requires_different_generic_type_params.ron

Failed in:
hugr_passes::const_fold::ConstantFoldPass::run takes 0 generic types instead of 1, in /home/runner/work/hugr/hugr/PR_BRANCH/hugr-passes/src/const_fold.rs:91
hugr_passes::RemoveDeadFuncsPass::run takes 0 generic types instead of 1, in /home/runner/work/hugr/hugr/PR_BRANCH/hugr-passes/src/dead_funcs.rs:88
hugr_passes::MonomorphizePass::run takes 0 generic types instead of 1, in /home/runner/work/hugr/hugr/PR_BRANCH/hugr-passes/src/monomorphize.rs:264
hugr_passes::untuple::UntuplePass::run takes 0 generic types instead of 1, in /home/runner/work/hugr/hugr/PR_BRANCH/hugr-passes/src/untuple.rs:129
hugr_passes::UntuplePass::run takes 0 generic types instead of 1, in /home/runner/work/hugr/hugr/PR_BRANCH/hugr-passes/src/untuple.rs:129
hugr_passes::replace_types::ReplaceTypes::run takes 0 generic types instead of 1, in /home/runner/work/hugr/hugr/PR_BRANCH/hugr-passes/src/replace_types.rs:444
hugr_passes::ReplaceTypes::run takes 0 generic types instead of 1, in /home/runner/work/hugr/hugr/PR_BRANCH/hugr-passes/src/replace_types.rs:444

--- failure module_missing: pub module removed or renamed ---

Description:
A publicly-visible module cannot be imported by its prior path. A `pub use` may have been removed, or the module may have been renamed, removed, or made non-public.
      ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
     impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.40.0/src/lints/module_missing.ron

Failed in:
mod hugr_passes::validation, previously in file /home/runner/work/hugr/hugr/BASELINE_BRANCH/hugr-passes/src/validation.rs:1

codecov · 2025-01-29T12:49:08Z

Codecov Report

Attention: Patch coverage is 89.45455% with 29 lines in your changes missing coverage. Please review.

Project coverage is 83.26%. Comparing base (195f30c) to head (a597008).
Report is 3 commits behind head on release-rs-v0.16.0.

Files with missing lines	Patch %	Lines
hugr-passes/src/composable.rs	88.77%	17 Missing and 4 partials ⚠️
hugr-passes/src/monomorphize.rs	77.77%	1 Missing and 3 partials ⚠️
hugr-passes/src/replace_types.rs	90.47%	1 Missing and 1 partial ⚠️
hugr-passes/src/const_fold.rs	75.00%	0 Missing and 1 partial ⚠️
hugr-passes/src/dead_funcs.rs	95.65%	1 Missing ⚠️

Additional details and impacted files

@@                  Coverage Diff                   @@
##           release-rs-v0.16.0    #1895      +/-   ##
======================================================
+ Coverage               83.15%   83.26%   +0.10%     
======================================================
  Files                     218      218              
  Lines                   41923    41948      +25     
  Branches                38101    38153      +52     
======================================================
+ Hits                    34861    34926      +65     
+ Misses                   5257     5217      -40     
  Partials                 1805     1805

Flag	Coverage Δ
python	`85.42% <ø> (+0.02%)`	⬆️
rust	`83.04% <89.45%> (+0.11%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

…ing validation

…er Dataflow

acl-cqc · 2025-03-14T14:33:12Z

hugr-passes/src/composable.rs

+
+    fn sequence(
+        self,
+        other: impl ComposablePass<Err = Self::Err>,


The other way to do this is

fn sequence_before<P2: ComposablePass>(self, other: P2) -> impl ComposablePass<Err = Self::Err> where P2::Err : Into<Self::Err>

(and similarly sequence_after) which avoids what may be a common-case use of map_err (i.e. with Into).

However, Infallible doesn't implement Into, so it's probably not much of a win. (I could add sequence_infallible(self, other: ComposablePass<Err=Infallible>) I guess, but would then want that before/after versions of that too)

cqc-alec

Looks good! I do think some more docs would be good. Regarding the suggestion to add add_entry_point -- does it mean to apply the pass to everything below that point in the hierarchy but not above it? I think that sounds reasonable.

hugr-passes/src/composable.rs

acl-cqc · 2025-04-15T12:33:25Z

The tuple-targets for ComposablePass carry the risk of overcomplicating the API. This is something we expect users to call regularly, so adding arbitrary parameter tupleing may feel unexpected.

@aborgna-q do you mean impl ComposablePass for (A,B)? I have removed these now in favour of a hidden struct as that let me do some magic with error traits and phantomdata.

acl-cqc · 2025-04-15T15:19:27Z

@doug-q @aborgna-q @cqc-alec this has changed quite a bit since Alec approved - as new passes have been added to main, requiring generalization here to handle passes with results. Could (any) one of you please re-review? I've also added a trait for combining two errors (allows combining two methods), and added an "IfThen" pass which needs the first result to return a bool.

I contemplate adding to the trait

fn map<T>(x: impl Fn(Self::Result) -> T) -> impl ComposablePass<Result=T, Error=Self::Error> {...} // impl provided

/// Just like [Self::then] but allows errors to be combined the other way around
fn then2<R2, E: CombineErrors<R2,Self::Error>>(other: ComposablePass<Result=R2, Error=E>) -> impl ComposablePass<Result=(Self::Result,R2), Error=E> {...}

But I guess there is no immediate need - these would be non-breaking if we added them later.

…posable_pass

acl-cqc · 2025-04-16T11:03:26Z

hugr-passes/src/composable.rs

+
+    /// Returns a [ComposablePass] that does "`self` then `other`", so long as
+    /// `other::Err` can be combined with ours.
+    fn then<P: ComposablePass, E: ErrorCombiner<Self::Error, P::Error>>(


There's a possible method then2 that requires E: ErrorCombiner<P::Error, Self::Error> i.e. that would allow when our error-type maps into P's (as opposed to then which requires the other way around).

However, both ways also allow returning an Either error and then2 would reverse that too (you probably wouldn't bother using then2 if you wanted an Either but). So then I wonder, should we also reverse the order of results...and the doubt as to what's the right API has lead me to avoid providing then2 entirely. But I could definitely be persuaded....

(Sadly we can't allow ErrorCombiner to work for both <A, B: Into<A>> and <A: Into<B>, B> because that would provide two conflicting impls when A and B are the same type :-( :-( )

cqc-alec · 2025-04-17T08:27:56Z

hugr-passes/src/composable.rs

+    }
+}
+
+// Note: in the short term two we could wish for more impls:


Suggested change

// Note: in the short term two we could wish for more impls:

// Note: in the short term we could wish for more impls:

aborgna-q

Nice!

aborgna-q · 2025-04-22T13:20:52Z

hugr-passes/src/composable.rs

+    fn from_second(b: B) -> Self;
+}
+
+impl<A: Error, B: Into<A>> ErrorCombiner<A, B> for A {


It'd be nice to have

impl<E: Error, A: Into<E>, B: Into<E>> ErrorCombiner<A, B> for E

instead, which covers the Either impl.
But that breaks the tests below as it doesn't allow us to have ErrorCombiner<A, Infallible> -.-

FTR...AFAICS Either does not implement From from either left or right, e.g. I get two errors here (one on each .into()):

fn foo<A,B>(a:A, b:B, which: bool) -> Either<A,B> { if which {a.into()} else {b.into()} }

aborgna-q · 2025-04-22T13:23:05Z

hugr-passes/src/composable.rs

+// Note: in the short term we could wish for two more impls:
+//   impl<E:Error> ErrorCombiner<Infallible, E> for E
+//   impl<E:Error> ErrorCombiner<E, Infallible> for E
+// however, these aren't possible as they conflict with
+//   impl<A, B:Into<A>> ErrorCombiner<A,B> for A
+// when A=E=Infallible, boo :-(.
+// However this will become possible, indeed automatic, when Infallible is replaced
+// by ! (never_type) as (unlike Infallible) ! converts Into anything


This is covered by impl<A: Error, B: Error> ErrorCombiner<A, B> for Either<A, B>, no?
It is required to compose DeadCodeElimPass in test_then.

Currently We have several "passes": monomorphization, dead function removal, constant folding. Each has its own code to allow setting a validation level (before and after that pass). This PR adds the ability chain (sequence) passes;, and to add validation before+after any pass or sequence; and commons up validation code. The top-level `constant_fold_pass` (etc.) functions are left as wrappers that do a single pass with validation only in test. I've left ConstFoldPass as always including DCE, but an alternative could be to return a sequence of the two - ATM that means a tuple `(ConstFoldPass, DeadCodeElimPass)`. I also wondered about including a method `add_entry_point` in ComposablePass (e.g. for ConstFoldPass, that means `with_inputs` but no inputs, i.e. all Top). I feel this is not applicable to *all* passes, but near enough. This could be done in a later PR but `add_entry_point` would need a no-op default for that to be a non-breaking change. So if we wouldn't be happy with the no-op default then I could just add it here... Finally...docs are extremely minimal ATM (this is hugr-passes), I am hoping that most of this is reasonably obvious (it doesn't really do a lot!), but please flag anything you think is particularly in need of a doc comment! BREAKING CHANGE: quite a lot of calls to current pass routines will break, specific cases include (a) `with_validation_level` should be done by wrapping a ValidatingPass around the receiver; (b) XXXPass::run() requires `use ...ComposablePass` (however, such calls will cease to do any validation). closes #1832

acl-cqc added 10 commits January 28, 2025 14:29

Basic composable passes: MapErr, (a,b) for sequence, Validating

f63c3d1

Add validate_if_test, rename ComposablePass::E -> Err

f3a0cb1

default to cfg! extension_inference

cac9b14

Make composable public, re-export only ComposablePass

d64c229

Update const_fold...add_entry_point still a TODO

fccb130

Remove add_entry_point....plan to revert later

c7b93ac

Update remove_dead_funcs

fbd1304

BREAKING: rm monomorphize, rename monomorphize_ref -> monomorphize

7f27dc3

Update monomorphize

fba2042

Remove hugr-passes::validation

0f09a49

acl-cqc added 2 commits January 29, 2025 12:39

Merge remote-tracking branch 'origin/main' into acl/composable_pass

f7f3493

Do not validate extensions in tests :-(

c6f446b

acl-cqc added 7 commits January 29, 2025 13:00

MonomorphizePass: don't derive Default; tests use monomorphize restor…

dab7039

…ing validation

Merge 'origin/main' into acl/composable_pass

6f4f1b7

Test sequencing, deriving Clone+PartialEq for ConstFoldError

7532b13

And test validation

5b94749

Driveby: no need for ConstFoldContext to implement Deref - from earli…

2487143

…er Dataflow

redundant missing docs

395b8c3

a few tweaks

c3e2e17

acl-cqc commented Mar 14, 2025

View reviewed changes

fmt

c8653d0

acl-cqc marked this pull request as ready for review March 14, 2025 14:40

acl-cqc requested a review from a team as a code owner March 14, 2025 14:40

acl-cqc requested a review from cqc-alec March 14, 2025 14:40

cqc-alec reviewed Mar 14, 2025

View reviewed changes

hugr-passes/src/composable.rs Show resolved Hide resolved

hugr-passes/src/composable.rs Show resolved Hide resolved

hugr-passes/src/composable.rs Outdated Show resolved Hide resolved

acl-cqc added 2 commits March 14, 2025 18:31

sequence -> then, add note about Infallible

bbbcd6f

rename Err -> Error

80cedfc

cqc-alec approved these changes Mar 17, 2025

View reviewed changes

Merge remote-tracking branch 'origin/main' into acl/composable_pass

1a9d28b

acl-cqc added 3 commits April 15, 2025 14:26

test_sequence => test_then, both orders

9d4b478

IfTrueThen=>IfThen, test; derive PartialEq for UntupleResult

58b7326

imports wtf git

8a1e4c5

acl-cqc added 3 commits April 15, 2025 17:26

Fix all-features

5daab98

Merge remote-tracking branch 'origin/main' into acl/composable_pass

4bbe0a6

Merge remote-tracking branch 'origin/release-rs-v0.16.0' into acl/com…

0fc3ed6

…posable_pass

acl-cqc requested a review from cqc-alec April 16, 2025 10:51

acl-cqc changed the base branch from main to release-rs-v0.16.0 April 16, 2025 10:51

acl-cqc added 2 commits April 16, 2025 11:56

tidy imports, correct comment

75fc36a

inline change_type_then_untup

b7b9569

acl-cqc commented Apr 16, 2025

View reviewed changes

acl-cqc added 2 commits April 16, 2025 16:37

clippy

c543be3

Fix all-features by disabling extension validation

662d765

cqc-alec approved these changes Apr 17, 2025

View reviewed changes

acl-cqc mentioned this pull request Apr 22, 2025

feat!: Add ReplaceBoolPass CQCL/tket2#854

Merged

comment typo

a597008

aborgna-q approved these changes Apr 22, 2025

View reviewed changes

acl-cqc added this pull request to the merge queue Apr 22, 2025

Merged via the queue into release-rs-v0.16.0 with commit 89c2680 Apr 22, 2025
25 checks passed

acl-cqc deleted the acl/composable_pass branch April 22, 2025 16:22

This was referenced May 7, 2025

chore: release v0.20.0 #2162

Closed

chore: release v0.20.0 #2181

Closed

hugrbot mentioned this pull request May 14, 2025

chore: release v0.21.0 #2220

Closed

This was referenced May 29, 2025

chore: release v0.21.0 #2271

Closed

chore: release v0.20.1 #2273

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat!: ComposablePass trait allowing sequencing and validation #1895

feat!: ComposablePass trait allowing sequencing and validation #1895

Uh oh!

acl-cqc commented Jan 28, 2025 •

edited

Loading

Uh oh!

hugrbot commented Jan 28, 2025 •

edited

Loading

Uh oh!

codecov bot commented Jan 29, 2025 •

edited

Loading

Uh oh!

acl-cqc Mar 14, 2025 •

edited

Loading

Uh oh!

cqc-alec left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

acl-cqc commented Apr 15, 2025

Uh oh!

acl-cqc commented Apr 15, 2025 •

edited

Loading

Uh oh!

acl-cqc Apr 16, 2025

Uh oh!

cqc-alec Apr 17, 2025

Uh oh!

aborgna-q left a comment

Uh oh!

aborgna-q Apr 22, 2025

Uh oh!

acl-cqc Apr 22, 2025

Uh oh!

aborgna-q Apr 22, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

	// Note: in the short term two we could wish for more impls:
	// Note: in the short term we could wish for more impls:

feat!: ComposablePass trait allowing sequencing and validation #1895

feat!: ComposablePass trait allowing sequencing and validation #1895

Uh oh!

Conversation

acl-cqc commented Jan 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hugrbot commented Jan 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Jan 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

acl-cqc Mar 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cqc-alec left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

acl-cqc commented Apr 15, 2025

Uh oh!

acl-cqc commented Apr 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

acl-cqc Apr 16, 2025

Choose a reason for hiding this comment

Uh oh!

cqc-alec Apr 17, 2025

Choose a reason for hiding this comment

Uh oh!

aborgna-q left a comment

Choose a reason for hiding this comment

Uh oh!

aborgna-q Apr 22, 2025

Choose a reason for hiding this comment

Uh oh!

acl-cqc Apr 22, 2025

Choose a reason for hiding this comment

Uh oh!

aborgna-q Apr 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

acl-cqc commented Jan 28, 2025 •

edited

Loading

hugrbot commented Jan 28, 2025 •

edited

Loading

codecov bot commented Jan 29, 2025 •

edited

Loading

acl-cqc Mar 14, 2025 •

edited

Loading

acl-cqc commented Apr 15, 2025 •

edited

Loading

aborgna-q Apr 22, 2025 •

edited

Loading