Skip to content

Conversation

@apoelstra
Copy link
Member

This PR is a series of commits which cleans up the expression parsing module. After the last couple PRs, which substantially rewrote the parser and introduce a new parsing-error module, we can get rid of many uses of the Error::Unexpected variant and its constructor, the errstr function.

This PR should have no visible effects, and does not even change any algorithms. The next one will return to the process of rewriting the expression parser, by replacing the recursive Tree type with a non-recursive one.

Will post benchmarks once they are done.

Improves the typing of the errors (and the error messages themselves) by
eliminating another instance of `errstr`.

Saves another instance of `errstr` by removing the unused `unary`
method. There are now 5 left in the entire library :).

Strictly speaking, we ought to do a deprecation cycle on these. But
because we are changing the `expression` module so thoroughly, and we
don't expect any users are directly using it, it doesn't seem worth the
difficult (impossible?) task of preserving the old API for the sake of
deprecation messages.
This is a big diff but it's almost entirely mechanical. Replaces the old
expression::terminal method with two new `verify_terminal_parent` and
`verify_terminal` methods, which do the appropriate checks and return a
strongly-typed error.

Does not directly eliminate any instances of errstr or Unexpected
(though the next commit will), but it does eliminate a large class of
them: now when you try to parse an expression with a bad number of
children, e.g. and_v with 3 children, you will get an error message that
says this rather than an opaque "Unexected(<<and_v>>)" or whatever you
get now.

It does reduce the semantic::PolicyError type to a single variant, which
has no information and is only used to indicate that the entailment
calculation failed. This can be replaced by an Option (and will be, in
the next commit). It also eliminates some uses of concrete::PolicyError,
but the variants are still used by the absurd Policy::is_valid method,
so we have to keep them for now.

Also, you may notice that this commit and others have a ton of calls to
.map_err. I apologize for this. But when we change error types so that
parsing returns a string-parsing-specific error rather than the giant
Error enum, these should mostly go away.
All semantic errors were actually parsing errors (except for the
entailment failure mode, which was better represented as an option). Now
that we are strongly-typing parsing errors we do not need this enum.
This eliminates another instance of `errstr` and provides well-typed
errors when parsing locktimes and threshold values.

It copies the AbsoluteLockTime and RelativeLockTime error variants from
the main Error enum into ParseError. The old copies are now used only in
script decoding, and will be removed in a later PR when we give script
decoding its own error.
This gets rid of several more instances of errstr.
The private function `with_huffman_tree` can only fail if it is given an
empty input. In both places we call it, we unwrap the result because we
know this is impossible.

Instead, just change the function to panic internally instead of
returning an error we're just going to unwrap.

Since the error was constructed with errstr, this gets rid of a call to
errstr.
When we are parsing keys using from_str, use the appropriate variant of
ParseError.

Also there was one place where we were taking an Error, converting it to
a string, then wrapping in Error::Unexpected. I suspect this was a
rebase mistake or something like that. Just get rid of the conversion.

There are now 2 instances of Error::Unexpected in the codebase and one
instance of errstr.
This gets rid of the horrible `errstr` function :).
In miniscript and in policy we had near-identical logic dealing with :
and @ separators on certain nodes. The differences were:

* In Miniscript we had special handling for aliases, where we would
  synthetically munge the wrappers (characters before ':'). This was
  unnecessary since we can just handle the aliases directly. (Because
  of our munging code, we did some extra error-checking to ensure that
  a PkK fragment always fits into a Check. It does. This checking is
  completely unnecessary.)
* In Policy we forbade the @ character if we were outside of an Or
  context. Also unnecessary. The @ character does not appear in any
  other fragment, so the "unknown fragment" error is already sufficient.

Removes two variants from the giant Error enum.
@apoelstra
Copy link
Member Author

Benchmarks with this PR (see #775 for previous numbers):

Benchmarks with 778

test benchmarks::parse_descriptor_balanced_segwit_a_0         ... bench:         544.61 ns/iter (+/- 12.82)
test benchmarks::parse_descriptor_balanced_segwit_b_1         ... bench:       6,629.07 ns/iter (+/- 45.20)
test benchmarks::parse_descriptor_balanced_segwit_c_10        ... bench:      70,120.73 ns/iter (+/- 1,685.72)
test benchmarks::parse_descriptor_balanced_segwit_d_20        ... bench:     140,782.60 ns/iter (+/- 2,387.76)
test benchmarks::parse_descriptor_balanced_segwit_e_40        ... bench:     282,633.23 ns/iter (+/- 6,204.65)
test benchmarks::parse_descriptor_balanced_segwit_f_60        ... bench:     424,342.15 ns/iter (+/- 8,569.58)
test benchmarks::parse_descriptor_balanced_segwit_g_80        ... bench:     567,367.60 ns/iter (+/- 2,751.23)
test benchmarks::parse_descriptor_balanced_segwit_h_90        ... bench:     639,606.20 ns/iter (+/- 16,083.55)
test benchmarks::parse_descriptor_balanced_segwit_thresh_a_1  ... bench:       6,634.88 ns/iter (+/- 125.62)
test benchmarks::parse_descriptor_balanced_segwit_thresh_b_10 ... bench:      75,322.15 ns/iter (+/- 1,735.03)
test benchmarks::parse_descriptor_balanced_segwit_thresh_c_20 ... bench:     150,571.65 ns/iter (+/- 1,933.92)
test benchmarks::parse_descriptor_balanced_segwit_thresh_d_40 ... bench:     303,809.17 ns/iter (+/- 3,166.47)
test benchmarks::parse_descriptor_balanced_segwit_thresh_e_60 ... bench:     456,862.00 ns/iter (+/- 7,290.80)
test benchmarks::parse_descriptor_balanced_segwit_thresh_f_80 ... bench:     609,439.60 ns/iter (+/- 3,358.01)
test benchmarks::parse_descriptor_balanced_segwit_thresh_g_90 ... bench:     706,377.30 ns/iter (+/- 23,088.26)
test benchmarks::parse_descriptor_deep_segwit_a_0             ... bench:         560.75 ns/iter (+/- 18.05)
test benchmarks::parse_descriptor_deep_segwit_b_1             ... bench:       6,649.21 ns/iter (+/- 42.06)
test benchmarks::parse_descriptor_deep_segwit_c_10            ... bench:      70,690.88 ns/iter (+/- 1,494.85)
test benchmarks::parse_descriptor_deep_segwit_d_20            ... bench:     143,412.96 ns/iter (+/- 1,221.54)
test benchmarks::parse_descriptor_deep_segwit_e_40            ... bench:     293,314.05 ns/iter (+/- 3,162.16)
test benchmarks::parse_descriptor_deep_segwit_f_60            ... bench:     446,141.60 ns/iter (+/- 8,750.10)
test benchmarks::parse_descriptor_deep_segwit_g_80            ... bench:     604,737.40 ns/iter (+/- 8,938.87)
test benchmarks::parse_descriptor_deep_segwit_h_90            ... bench:     686,288.20 ns/iter (+/- 8,319.25)
test benchmarks::parse_descriptor_deep_segwit_thresh_a_1      ... bench:       6,623.19 ns/iter (+/- 90.72)
test benchmarks::parse_descriptor_deep_segwit_thresh_b_10     ... bench:      76,314.25 ns/iter (+/- 344.57)
test benchmarks::parse_descriptor_deep_segwit_thresh_c_20     ... bench:     155,510.10 ns/iter (+/- 2,893.96)
test benchmarks::parse_descriptor_deep_segwit_thresh_d_40     ... bench:     319,904.17 ns/iter (+/- 6,282.61)
test benchmarks::parse_descriptor_deep_segwit_thresh_e_60     ... bench:     488,473.05 ns/iter (+/- 9,353.83)
test benchmarks::parse_descriptor_deep_segwit_thresh_f_80     ... bench:     663,055.90 ns/iter (+/- 4,961.75)
test benchmarks::parse_descriptor_deep_segwit_thresh_g_90     ... bench:     752,448.00 ns/iter (+/- 14,050.81)
test benchmarks::parse_descriptor_tr_bigtree_a_1              ... bench:      13,552.07 ns/iter (+/- 326.65)
test benchmarks::parse_descriptor_tr_bigtree_b_2              ... bench:      21,139.16 ns/iter (+/- 439.49)
test benchmarks::parse_descriptor_tr_bigtree_c_5              ... bench:      43,552.84 ns/iter (+/- 953.23)
test benchmarks::parse_descriptor_tr_bigtree_d_10             ... bench:      80,783.83 ns/iter (+/- 398.38)
test benchmarks::parse_descriptor_tr_bigtree_e_20             ... bench:     155,040.00 ns/iter (+/- 2,651.30)
test benchmarks::parse_descriptor_tr_bigtree_f_50             ... bench:     379,000.40 ns/iter (+/- 5,207.57)
test benchmarks::parse_descriptor_tr_bigtree_g_100            ... bench:     755,831.60 ns/iter (+/- 9,391.77)
test benchmarks::parse_descriptor_tr_bigtree_h_200            ... bench:   1,519,807.00 ns/iter (+/- 21,208.81)
test benchmarks::parse_descriptor_tr_bigtree_i_500            ... bench:   3,887,346.50 ns/iter (+/- 53,880.88)
test benchmarks::parse_descriptor_tr_bigtree_j_1000           ... bench:   7,959,709.10 ns/iter (+/- 53,128.03)
test benchmarks::parse_descriptor_tr_bigtree_k_2000           ... bench:  15,822,429.40 ns/iter (+/- 267,708.81)
test benchmarks::parse_descriptor_tr_bigtree_l_5000           ... bench:  39,705,637.70 ns/iter (+/- 602,615.93)
test benchmarks::parse_descriptor_tr_bigtree_m_10000          ... bench:  80,692,465.30 ns/iter (+/- 1,603,950.48)
test benchmarks::parse_descriptor_tr_deep_bigtree_a_1         ... bench:      13,540.98 ns/iter (+/- 483.34)
test benchmarks::parse_descriptor_tr_deep_bigtree_b_2         ... bench:      21,097.03 ns/iter (+/- 398.22)
test benchmarks::parse_descriptor_tr_deep_bigtree_c_5         ... bench:      43,379.65 ns/iter (+/- 763.85)
test benchmarks::parse_descriptor_tr_deep_bigtree_d_10        ... bench:      80,526.60 ns/iter (+/- 569.58)
test benchmarks::parse_descriptor_tr_deep_bigtree_e_20        ... bench:     154,872.18 ns/iter (+/- 2,440.73)
test benchmarks::parse_descriptor_tr_deep_bigtree_f_50        ... bench:     380,018.85 ns/iter (+/- 5,786.23)
test benchmarks::parse_descriptor_tr_deep_bigtree_g_100       ... bench:     749,391.00 ns/iter (+/- 10,336.66)
test benchmarks::parse_descriptor_tr_deep_bigtree_h_128       ... bench:     960,486.00 ns/iter (+/- 23,908.90)
test benchmarks::parse_descriptor_tr_deep_oneleaf_a_1         ... bench:      13,627.45 ns/iter (+/- 155.29)
test benchmarks::parse_descriptor_tr_deep_oneleaf_b_10        ... bench:      87,550.09 ns/iter (+/- 1,155.28)
test benchmarks::parse_descriptor_tr_deep_oneleaf_c_20        ... bench:     179,011.72 ns/iter (+/- 1,419.09)
test benchmarks::parse_descriptor_tr_deep_oneleaf_d_50        ... bench:     446,604.55 ns/iter (+/- 4,182.67)
test benchmarks::parse_descriptor_tr_deep_oneleaf_e_100       ... bench:     935,830.00 ns/iter (+/- 9,373.02)
test benchmarks::parse_descriptor_tr_deep_oneleaf_f_200       ... bench:   2,028,258.80 ns/iter (+/- 31,256.08)
test benchmarks::parse_descriptor_tr_oneleaf_a_1              ... bench:      13,641.24 ns/iter (+/- 94.69)
test benchmarks::parse_descriptor_tr_oneleaf_b_10             ... bench:      87,176.53 ns/iter (+/- 846.62)
test benchmarks::parse_descriptor_tr_oneleaf_c_20             ... bench:     175,789.64 ns/iter (+/- 3,596.59)
test benchmarks::parse_descriptor_tr_oneleaf_d_50             ... bench:     431,507.65 ns/iter (+/- 8,758.08)
test benchmarks::parse_descriptor_tr_oneleaf_e_100            ... bench:     881,033.40 ns/iter (+/- 15,511.56)
test benchmarks::parse_descriptor_tr_oneleaf_f_200            ... bench:   1,815,389.50 ns/iter (+/- 21,167.29)
test benchmarks::parse_descriptor_tr_oneleaf_g_500            ... bench:   4,678,999.50 ns/iter (+/- 35,057.19)
test benchmarks::parse_descriptor_tr_oneleaf_h_1000           ... bench:   9,621,959.20 ns/iter (+/- 77,829.97)
test benchmarks::parse_descriptor_tr_oneleaf_i_2000           ... bench:  19,497,634.50 ns/iter (+/- 91,959.28)
test benchmarks::parse_descriptor_tr_oneleaf_j_5000           ... bench:  54,216,270.00 ns/iter (+/- 363,527.41)
test benchmarks::parse_descriptor_tr_oneleaf_k_10000          ... bench: 115,250,372.90 ns/iter (+/- 588,768.19)
test benchmarks::parse_expression_balanced_a_0                ... bench:         156.69 ns/iter (+/- 6.25)
test benchmarks::parse_expression_balanced_b_1                ... bench:         556.51 ns/iter (+/- 6.30)
test benchmarks::parse_expression_balanced_c_2                ... bench:         952.13 ns/iter (+/- 18.08)
test benchmarks::parse_expression_balanced_d_5                ... bench:       2,290.57 ns/iter (+/- 49.22)
test benchmarks::parse_expression_balanced_e_10               ... bench:       4,759.23 ns/iter (+/- 65.25)
test benchmarks::parse_expression_balanced_f_20               ... bench:       9,833.07 ns/iter (+/- 165.60)
test benchmarks::parse_expression_balanced_g_50               ... bench:      24,468.55 ns/iter (+/- 588.51)
test benchmarks::parse_expression_balanced_h_100              ... bench:      48,717.05 ns/iter (+/- 1,480.83)
test benchmarks::parse_expression_balanced_i_200              ... bench:      96,375.19 ns/iter (+/- 2,076.64)
test benchmarks::parse_expression_balanced_j_500              ... bench:     240,661.28 ns/iter (+/- 6,397.98)
test benchmarks::parse_expression_balanced_k_1000             ... bench:     486,693.05 ns/iter (+/- 14,785.87)
test benchmarks::parse_expression_balanced_l_2000             ... bench:   1,009,295.10 ns/iter (+/- 165,908.63)
test benchmarks::parse_expression_balanced_m_5000             ... bench:   2,481,216.70 ns/iter (+/- 186,711.76)
test benchmarks::parse_expression_balanced_n_10000            ... bench:   5,183,672.65 ns/iter (+/- 853,395.72)
test benchmarks::parse_expression_deep_a_0                    ... bench:         163.86 ns/iter (+/- 7.32)
test benchmarks::parse_expression_deep_b_1                    ... bench:         591.60 ns/iter (+/- 6.71)
test benchmarks::parse_expression_deep_c_2                    ... bench:         937.90 ns/iter (+/- 15.59)
test benchmarks::parse_expression_deep_d_5                    ... bench:       2,282.76 ns/iter (+/- 21.38)
test benchmarks::parse_expression_deep_e_10                   ... bench:       4,730.56 ns/iter (+/- 60.88)
test benchmarks::parse_expression_deep_f_20                   ... bench:       9,621.05 ns/iter (+/- 82.22)
test benchmarks::parse_expression_deep_g_50                   ... bench:      23,602.95 ns/iter (+/- 325.39)
test benchmarks::parse_expression_deep_h_100                  ... bench:      46,756.14 ns/iter (+/- 898.57)
test benchmarks::parse_expression_deep_i_200                  ... bench:      92,550.43 ns/iter (+/- 971.18)
test benchmarks::parse_expression_deep_j_300                  ... bench:     138,381.27 ns/iter (+/- 2,073.32)
test benchmarks::parse_expression_deep_j_400                  ... bench:     185,200.36 ns/iter (+/- 1,325.20)

You can see from comparing to the previous PR that the numbers are basically identical.

Copy link
Member

@sanket1729 sanket1729 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 33a60e2

Awesome. Thanks for the super helpful commit messages. I felt I was a part of the journey hunting down all the errstrs

@apoelstra apoelstra merged commit 733bedd into rust-bitcoin:master Nov 27, 2024
30 checks passed
@apoelstra apoelstra deleted the 2024-11--expression-3 branch November 27, 2024 15:57
heap-coder added a commit to heap-coder/rust-miniscript that referenced this pull request Sep 27, 2025
…and (nearly) eliminate `Unexpected`

33a60e29bd089cc14b9fd00f89e69903d2275a65 expression: pull MultiColon error into parsing logic, drop AtOutsideOr (Andrew Poelstra)
f7cb7013cc9bef027a853ffdb2c4744becdc1f07 expression: add "illegal and/or" for thresholds; drop errstr (Andrew Poelstra)
b3d1b171f9c3d13cf195dcea0c8a51545c4beff4 descriptor: eliminate several instances of Unexpected (Andrew Poelstra)
1467453d3999dcde2ac62ac82c61ca7fbc0b2a46 compiler: refactor out a call to errstr (Andrew Poelstra)
dd19874380e958132d541af4a74c5cca8f930cc2 expression: introduce "unknown name" error variant (Andrew Poelstra)
4ae5079ae3c9df0d80cabe6fc32fbaf0f9819cd7 expression: replace methods for parsing numbers and locktimes (Andrew Poelstra)
ee1505613693166beb8a11307d8fd1c3eca4fe48 policy: remove now-unused semantic::PolicyError type (Andrew Poelstra)
ce3d3e8b7b75a64df11700a13b1637670d0f70aa expression: replace most uses of `terminal` (Andrew Poelstra)
8423557514649c2403ffe5df18cf04d10ef6b32a expression: replace poorly-typed `binary` function with new one (Andrew Poelstra)

Pull request description:

  This PR is a series of commits which cleans up the expression parsing module. After the last couple PRs, which substantially rewrote the parser and introduce a new parsing-error module, we can get rid of many uses of the `Error::Unexpected` variant and its constructor, the `errstr` function.

  This PR should have no visible effects, and does not even change any algorithms. The next one will return to the process of rewriting the expression parser, by replacing the recursive `Tree` type with a non-recursive one.

  Will post benchmarks once they are done.

ACKs for top commit:
  sanket1729:
    ACK 33a60e29bd089cc14b9fd00f89e69903d2275a65

Tree-SHA512: 09927b83f84baa3593af22e95b84aa38f65851e6e005fb53f02cbb23edc7e2275345727399d70a252ccd1f8462707666b91fc043f613d2998fa8a51204525a9e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants