expression: implement parsing of policies and miniscripts non-recursively #784

apoelstra · 2024-12-17T20:51:33Z

This brings us to the end of my cached "rewrite expression module" branch, by making expression parsing non-recursive. The diff here is actually reasonably short but it's pretty invasive. Hopefully the new fuzz test from the last PR is enough to convince us of correctness.

I have some more work to do in two directions:

Breaking up the two Policy types into a structs with inner enums (or possibly just a flat tree type), which will allow us to attach metadata to nodes and trees; after this, I will introduce the ValidationParams structure and start moving sanity/validation/parsing logic into there as suggested in Can we eliminate the `Ctx` parameter? #723. I have some incomplete commits in this direction but I probably want to redo them with lessons learned from the Taproot work.
Replacing the TapTree enum with a non-recursive type, which will let us replace the taproot iteration API with a more natural one (as well as getting some performance benefit). I have more-complete work on this, but I'm getting a bit hamstrung right now handling the difference between MiniscriptKey and ToPublicKey in this specialization-less language.

All of the terminals in Miniscript can be constructed infallibly. Add constructors for this so we don't need to go through Miniscript::from_ast and deal with an impossible-to-hit error path.

This also drops the FromTree impls for Terminal and Arc<Terminal>. As argued elsewhere, the Terminal types are not first-class types and it doesn't make sense to allow them to be constructed in elaborate ways. The benchmarks are pretty noisy but appear to show a pretty big speedup. On master: test expression::benches::parse_tree ... bench: 885.43 ns/iter (+/- 412.97) test expression::benches::parse_tree_deep ... bench: 1,951.10 ns/iter (+/- 845.66) test miniscript::benches::parse_segwit0 ... bench: 9,571.35 ns/iter (+/- 565.98) test miniscript::benches::parse_segwit0_deep ... bench: 25,571.58 ns/iter (+/- 468.93) On this commit: test expression::benches::parse_tree ... bench: 1,020.52 ns/iter (+/- 139.58) test expression::benches::parse_tree_deep ... bench: 1,632.75 ns/iter (+/- 176.35) test miniscript::benches::parse_segwit0 ... bench: 10,747.28 ns/iter (+/- 2,338.26) test miniscript::benches::parse_segwit0_deep ... bench: 19,090.65 ns/iter (+/- 1,040.62) So if you take the numbers at face value, that's a 33% speedup on the parse_segwit0_deep benchmark. Nice.

apoelstra · 2024-12-17T20:55:35Z

Benchmarks on master (copied from #775 (comment) )

Benchmarks on master

test benchmarks::parse_descriptor_balanced_segwit_a_0         ... bench:         465.89 ns/iter (+/- 13.02)
test benchmarks::parse_descriptor_balanced_segwit_b_1         ... bench:       6,597.31 ns/iter (+/- 61.55)
test benchmarks::parse_descriptor_balanced_segwit_c_10        ... bench:      68,402.39 ns/iter (+/- 441.03)
test benchmarks::parse_descriptor_balanced_segwit_d_20        ... bench:     137,767.80 ns/iter (+/- 624.76)
test benchmarks::parse_descriptor_balanced_segwit_e_40        ... bench:     275,717.27 ns/iter (+/- 1,546.82)
test benchmarks::parse_descriptor_balanced_segwit_f_60        ... bench:     414,038.60 ns/iter (+/- 2,082.34)
test benchmarks::parse_descriptor_balanced_segwit_g_80        ... bench:     554,599.20 ns/iter (+/- 4,152.20)
test benchmarks::parse_descriptor_balanced_segwit_h_90        ... bench:     623,262.80 ns/iter (+/- 4,691.03)
test benchmarks::parse_descriptor_balanced_segwit_thresh_a_1  ... bench:       6,589.24 ns/iter (+/- 48.19)
test benchmarks::parse_descriptor_balanced_segwit_thresh_b_10 ... bench:      72,866.36 ns/iter (+/- 492.22)
test benchmarks::parse_descriptor_balanced_segwit_thresh_c_20 ... bench:     146,935.67 ns/iter (+/- 712.95)
test benchmarks::parse_descriptor_balanced_segwit_thresh_d_40 ... bench:     294,765.77 ns/iter (+/- 2,196.87)
test benchmarks::parse_descriptor_balanced_segwit_thresh_e_60 ... bench:     443,131.05 ns/iter (+/- 3,384.32)
test benchmarks::parse_descriptor_balanced_segwit_thresh_f_80 ... bench:     592,657.30 ns/iter (+/- 4,791.92)
test benchmarks::parse_descriptor_balanced_segwit_thresh_g_90 ... bench:     666,919.80 ns/iter (+/- 5,135.12)
test benchmarks::parse_descriptor_deep_segwit_a_0             ... bench:         478.37 ns/iter (+/- 10.21)
test benchmarks::parse_descriptor_deep_segwit_b_1             ... bench:       6,567.93 ns/iter (+/- 56.41)
test benchmarks::parse_descriptor_deep_segwit_c_10            ... bench:      68,742.34 ns/iter (+/- 477.96)
test benchmarks::parse_descriptor_deep_segwit_d_20            ... bench:     137,875.35 ns/iter (+/- 1,030.64)
test benchmarks::parse_descriptor_deep_segwit_e_40            ... bench:     276,197.60 ns/iter (+/- 1,738.74)
test benchmarks::parse_descriptor_deep_segwit_f_60            ... bench:     413,451.15 ns/iter (+/- 2,683.51)
test benchmarks::parse_descriptor_deep_segwit_g_80            ... bench:     552,011.10 ns/iter (+/- 4,749.70)
test benchmarks::parse_descriptor_deep_segwit_h_90            ... bench:     622,394.10 ns/iter (+/- 4,594.77)
test benchmarks::parse_descriptor_deep_segwit_thresh_a_1      ... bench:       6,567.16 ns/iter (+/- 64.12)
test benchmarks::parse_descriptor_deep_segwit_thresh_b_10     ... bench:      73,577.97 ns/iter (+/- 192.60)
test benchmarks::parse_descriptor_deep_segwit_thresh_c_20     ... bench:     148,726.82 ns/iter (+/- 536.78)
test benchmarks::parse_descriptor_deep_segwit_thresh_d_40     ... bench:     298,703.57 ns/iter (+/- 1,246.14)
test benchmarks::parse_descriptor_deep_segwit_thresh_e_60     ... bench:     448,690.85 ns/iter (+/- 1,734.72)
test benchmarks::parse_descriptor_deep_segwit_thresh_f_80     ... bench:     600,598.70 ns/iter (+/- 14,983.91)
test benchmarks::parse_descriptor_deep_segwit_thresh_g_90     ... bench:     675,715.00 ns/iter (+/- 3,019.62)
test benchmarks::parse_descriptor_tr_bigtree_a_1              ... bench:      13,370.86 ns/iter (+/- 65.97)
test benchmarks::parse_descriptor_tr_bigtree_b_2              ... bench:      20,506.94 ns/iter (+/- 89.95)
test benchmarks::parse_descriptor_tr_bigtree_c_5              ... bench:      42,363.64 ns/iter (+/- 218.79)
test benchmarks::parse_descriptor_tr_bigtree_d_10             ... bench:      79,039.02 ns/iter (+/- 316.80)
test benchmarks::parse_descriptor_tr_bigtree_e_20             ... bench:     153,134.58 ns/iter (+/- 589.79)
test benchmarks::parse_descriptor_tr_bigtree_f_50             ... bench:     374,249.90 ns/iter (+/- 1,492.46)
test benchmarks::parse_descriptor_tr_bigtree_g_100            ... bench:     745,691.20 ns/iter (+/- 3,326.81)
test benchmarks::parse_descriptor_tr_bigtree_h_200            ... bench:   1,489,229.40 ns/iter (+/- 5,734.94)
test benchmarks::parse_descriptor_tr_bigtree_i_500            ... bench:   3,925,334.70 ns/iter (+/- 10,281.20)
test benchmarks::parse_descriptor_tr_bigtree_j_1000           ... bench:   7,935,760.10 ns/iter (+/- 19,353.37)
test benchmarks::parse_descriptor_tr_bigtree_k_2000           ... bench:  15,910,600.30 ns/iter (+/- 44,100.49)
test benchmarks::parse_descriptor_tr_bigtree_l_5000           ... bench:  39,977,807.80 ns/iter (+/- 147,594.49)
test benchmarks::parse_descriptor_tr_bigtree_m_10000          ... bench:  80,776,176.60 ns/iter (+/- 298,184.28)
test benchmarks::parse_descriptor_tr_deep_bigtree_a_1         ... bench:      13,373.61 ns/iter (+/- 81.58)
test benchmarks::parse_descriptor_tr_deep_bigtree_b_2         ... bench:      20,531.85 ns/iter (+/- 188.93)
test benchmarks::parse_descriptor_tr_deep_bigtree_c_5         ... bench:      42,375.48 ns/iter (+/- 254.94)
test benchmarks::parse_descriptor_tr_deep_bigtree_d_10        ... bench:      78,879.20 ns/iter (+/- 566.76)
test benchmarks::parse_descriptor_tr_deep_bigtree_e_20        ... bench:     154,271.45 ns/iter (+/- 1,588.20)
test benchmarks::parse_descriptor_tr_deep_bigtree_f_50        ... bench:     377,169.55 ns/iter (+/- 2,861.75)
test benchmarks::parse_descriptor_tr_deep_bigtree_g_100       ... bench:     744,043.10 ns/iter (+/- 5,380.24)
test benchmarks::parse_descriptor_tr_deep_bigtree_h_128       ... bench:     956,334.40 ns/iter (+/- 6,342.36)
test benchmarks::parse_descriptor_tr_deep_oneleaf_a_1         ... bench:      13,399.77 ns/iter (+/- 105.96)
test benchmarks::parse_descriptor_tr_deep_oneleaf_b_10        ... bench:      82,051.34 ns/iter (+/- 485.98)
test benchmarks::parse_descriptor_tr_deep_oneleaf_c_20        ... bench:     161,314.56 ns/iter (+/- 1,054.12)
test benchmarks::parse_descriptor_tr_deep_oneleaf_d_50        ... bench:     394,023.85 ns/iter (+/- 3,161.88)
test benchmarks::parse_descriptor_tr_deep_oneleaf_e_100       ... bench:     793,179.80 ns/iter (+/- 5,086.54)
test benchmarks::parse_descriptor_tr_deep_oneleaf_f_200       ... bench:   1,615,137.60 ns/iter (+/- 9,648.42)
test benchmarks::parse_descriptor_tr_oneleaf_a_1              ... bench:      13,444.67 ns/iter (+/- 107.97)
test benchmarks::parse_descriptor_tr_oneleaf_b_10             ... bench:      82,180.59 ns/iter (+/- 538.77)
test benchmarks::parse_descriptor_tr_oneleaf_c_20             ... bench:     161,952.91 ns/iter (+/- 1,220.03)
test benchmarks::parse_descriptor_tr_oneleaf_d_50             ... bench:     395,816.30 ns/iter (+/- 4,442.46)
test benchmarks::parse_descriptor_tr_oneleaf_e_100            ... bench:     794,898.10 ns/iter (+/- 3,452.93)
test benchmarks::parse_descriptor_tr_oneleaf_f_200            ... bench:   1,624,713.20 ns/iter (+/- 8,085.24)
test benchmarks::parse_descriptor_tr_oneleaf_g_500            ... bench:   4,127,597.60 ns/iter (+/- 12,541.62)
test benchmarks::parse_descriptor_tr_oneleaf_h_1000           ... bench:   8,393,876.20 ns/iter (+/- 27,310.88)
test benchmarks::parse_descriptor_tr_oneleaf_i_2000           ... bench:  16,971,642.50 ns/iter (+/- 55,966.49)
test benchmarks::parse_descriptor_tr_oneleaf_j_5000           ... bench:  48,622,186.60 ns/iter (+/- 166,228.02)
test benchmarks::parse_descriptor_tr_oneleaf_k_10000          ... bench: 101,239,307.00 ns/iter (+/- 248,620.43)
test benchmarks::parse_expression_balanced_a_0                ... bench:         158.13 ns/iter (+/- 2.52)
test benchmarks::parse_expression_balanced_b_1                ... bench:         431.17 ns/iter (+/- 6.79)
test benchmarks::parse_expression_balanced_c_2                ... bench:         788.64 ns/iter (+/- 12.22)
test benchmarks::parse_expression_balanced_d_5                ... bench:       1,938.73 ns/iter (+/- 14.29)
test benchmarks::parse_expression_balanced_e_10               ... bench:       4,112.92 ns/iter (+/- 53.97)
test benchmarks::parse_expression_balanced_f_20               ... bench:       8,169.31 ns/iter (+/- 131.74)
test benchmarks::parse_expression_balanced_g_50               ... bench:      20,087.98 ns/iter (+/- 315.16)
test benchmarks::parse_expression_balanced_h_100              ... bench:      40,489.81 ns/iter (+/- 313.45)
test benchmarks::parse_expression_balanced_i_200              ... bench:      80,639.21 ns/iter (+/- 679.48)
test benchmarks::parse_expression_balanced_j_500              ... bench:     200,178.08 ns/iter (+/- 3,613.59)
test benchmarks::parse_expression_balanced_k_1000             ... bench:     403,450.25 ns/iter (+/- 5,369.78)
test benchmarks::parse_expression_balanced_l_2000             ... bench:     802,836.05 ns/iter (+/- 9,533.49)
test benchmarks::parse_expression_balanced_m_5000             ... bench:   2,030,183.80 ns/iter (+/- 26,620.83)
test benchmarks::parse_expression_balanced_n_10000            ... bench:   4,159,970.60 ns/iter (+/- 59,766.04)
test benchmarks::parse_expression_deep_a_0                    ... bench:         143.35 ns/iter (+/- 4.26)
test benchmarks::parse_expression_deep_b_1                    ... bench:         425.64 ns/iter (+/- 12.74)
test benchmarks::parse_expression_deep_c_2                    ... bench:         767.49 ns/iter (+/- 15.56)
test benchmarks::parse_expression_deep_d_5                    ... bench:       1,903.38 ns/iter (+/- 23.07)
test benchmarks::parse_expression_deep_e_10                   ... bench:       4,050.42 ns/iter (+/- 76.06)
test benchmarks::parse_expression_deep_f_20                   ... bench:       7,877.37 ns/iter (+/- 119.96)
test benchmarks::parse_expression_deep_g_50                   ... bench:      19,092.16 ns/iter (+/- 58.22)
test benchmarks::parse_expression_deep_h_100                  ... bench:      38,052.99 ns/iter (+/- 266.18)
test benchmarks::parse_expression_deep_i_200                  ... bench:      75,396.99 ns/iter (+/- 636.95)
test benchmarks::parse_expression_deep_j_300                  ... bench:     114,219.85 ns/iter (+/- 519.24)
test benchmarks::parse_expression_deep_j_400                  ... bench:     151,819.67 ns/iter (+/- 862.04)

Benchmarks on this PR

test benchmarks::parse_descriptor_balanced_segwit_a_0         ... bench:         457.40 ns/iter (+/- 12.48)
test benchmarks::parse_descriptor_balanced_segwit_b_1         ... bench:       6,483.57 ns/iter (+/- 66.34)
test benchmarks::parse_descriptor_balanced_segwit_c_10        ... bench:      64,822.15 ns/iter (+/- 505.84)
test benchmarks::parse_descriptor_balanced_segwit_d_20        ... bench:     129,552.80 ns/iter (+/- 952.94)
test benchmarks::parse_descriptor_balanced_segwit_e_40        ... bench:     258,572.20 ns/iter (+/- 1,358.19)
test benchmarks::parse_descriptor_balanced_segwit_f_60        ... bench:     387,445.45 ns/iter (+/- 1,268.11)
test benchmarks::parse_descriptor_balanced_segwit_g_80        ... bench:     515,831.80 ns/iter (+/- 2,926.13)
test benchmarks::parse_descriptor_balanced_segwit_h_90        ... bench:     580,979.50 ns/iter (+/- 3,197.83)
test benchmarks::parse_descriptor_balanced_segwit_thresh_a_1  ... bench:       6,488.32 ns/iter (+/- 47.55)
test benchmarks::parse_descriptor_balanced_segwit_thresh_b_10 ... bench:      69,857.71 ns/iter (+/- 262.35)
test benchmarks::parse_descriptor_balanced_segwit_thresh_c_20 ... bench:     140,265.54 ns/iter (+/- 1,533.39)
test benchmarks::parse_descriptor_balanced_segwit_thresh_d_40 ... bench:     281,306.40 ns/iter (+/- 1,057.22)
test benchmarks::parse_descriptor_balanced_segwit_thresh_e_60 ... bench:     422,062.85 ns/iter (+/- 3,442.99)
test benchmarks::parse_descriptor_balanced_segwit_thresh_f_80 ... bench:     562,651.00 ns/iter (+/- 8,011.73)
test benchmarks::parse_descriptor_balanced_segwit_thresh_g_90 ... bench:     633,460.70 ns/iter (+/- 5,366.58)
test benchmarks::parse_descriptor_deep_segwit_a_0             ... bench:         454.21 ns/iter (+/- 8.32)
test benchmarks::parse_descriptor_deep_segwit_b_1             ... bench:       6,481.81 ns/iter (+/- 43.59)
test benchmarks::parse_descriptor_deep_segwit_c_10            ... bench:      64,808.69 ns/iter (+/- 313.69)
test benchmarks::parse_descriptor_deep_segwit_d_20            ... bench:     129,143.87 ns/iter (+/- 590.19)
test benchmarks::parse_descriptor_deep_segwit_e_40            ... bench:     257,487.23 ns/iter (+/- 789.07)
test benchmarks::parse_descriptor_deep_segwit_f_60            ... bench:     385,327.40 ns/iter (+/- 2,572.29)
test benchmarks::parse_descriptor_deep_segwit_g_80            ... bench:     513,680.70 ns/iter (+/- 2,675.24)
test benchmarks::parse_descriptor_deep_segwit_h_90            ... bench:     577,056.00 ns/iter (+/- 2,929.38)
test benchmarks::parse_descriptor_deep_segwit_thresh_a_1      ... bench:       6,486.83 ns/iter (+/- 32.42)
test benchmarks::parse_descriptor_deep_segwit_thresh_b_10     ... bench:      69,768.54 ns/iter (+/- 263.65)
test benchmarks::parse_descriptor_deep_segwit_thresh_c_20     ... bench:     139,282.61 ns/iter (+/- 522.24)
test benchmarks::parse_descriptor_deep_segwit_thresh_d_40     ... bench:     278,010.00 ns/iter (+/- 1,502.39)
test benchmarks::parse_descriptor_deep_segwit_thresh_e_60     ... bench:     416,151.85 ns/iter (+/- 1,808.83)
test benchmarks::parse_descriptor_deep_segwit_thresh_f_80     ... bench:     555,824.60 ns/iter (+/- 3,130.61)
test benchmarks::parse_descriptor_deep_segwit_thresh_g_90     ... bench:     625,157.60 ns/iter (+/- 3,856.96)
test benchmarks::parse_descriptor_tr_bigtree_a_1              ... bench:      13,184.69 ns/iter (+/- 77.47)
test benchmarks::parse_descriptor_tr_bigtree_b_2              ... bench:      20,461.84 ns/iter (+/- 89.96)
test benchmarks::parse_descriptor_tr_bigtree_c_5              ... bench:      41,680.07 ns/iter (+/- 182.11)
test benchmarks::parse_descriptor_tr_bigtree_d_10             ... bench:      77,539.73 ns/iter (+/- 326.50)
test benchmarks::parse_descriptor_tr_bigtree_e_20             ... bench:     149,303.55 ns/iter (+/- 598.79)
test benchmarks::parse_descriptor_tr_bigtree_f_50             ... bench:     363,439.75 ns/iter (+/- 1,803.04)
test benchmarks::parse_descriptor_tr_bigtree_g_100            ... bench:     718,394.00 ns/iter (+/- 6,094.70)
test benchmarks::parse_descriptor_tr_bigtree_h_200            ... bench:   1,435,519.50 ns/iter (+/- 6,567.69)
test benchmarks::parse_descriptor_tr_bigtree_i_500            ... bench:   3,579,644.10 ns/iter (+/- 13,669.47)
test benchmarks::parse_descriptor_tr_bigtree_j_1000           ... bench:   7,162,408.20 ns/iter (+/- 15,721.40)
test benchmarks::parse_descriptor_tr_bigtree_k_2000           ... bench:  14,312,083.70 ns/iter (+/- 31,811.42)
test benchmarks::parse_descriptor_tr_bigtree_l_5000           ... bench:  35,751,836.00 ns/iter (+/- 180,582.75)
test benchmarks::parse_descriptor_tr_bigtree_m_10000          ... bench:  71,327,926.10 ns/iter (+/- 295,680.41)
test benchmarks::parse_descriptor_tr_deep_bigtree_a_1         ... bench:      13,169.81 ns/iter (+/- 62.36)
test benchmarks::parse_descriptor_tr_deep_bigtree_b_2         ... bench:      20,415.43 ns/iter (+/- 106.49)
test benchmarks::parse_descriptor_tr_deep_bigtree_c_5         ... bench:      41,718.40 ns/iter (+/- 147.09)
test benchmarks::parse_descriptor_tr_deep_bigtree_d_10        ... bench:      77,427.01 ns/iter (+/- 356.07)
test benchmarks::parse_descriptor_tr_deep_bigtree_e_20        ... bench:     148,642.27 ns/iter (+/- 745.30)
test benchmarks::parse_descriptor_tr_deep_bigtree_f_50        ... bench:     361,024.55 ns/iter (+/- 1,472.38)
test benchmarks::parse_descriptor_tr_deep_bigtree_g_100       ... bench:     713,415.30 ns/iter (+/- 3,083.40)
test benchmarks::parse_descriptor_tr_deep_bigtree_h_128       ... bench:     912,120.00 ns/iter (+/- 5,070.63)
test benchmarks::parse_descriptor_tr_deep_oneleaf_a_1         ... bench:      13,163.66 ns/iter (+/- 71.89)
test benchmarks::parse_descriptor_tr_deep_oneleaf_b_10        ... bench:      79,852.90 ns/iter (+/- 413.95)
test benchmarks::parse_descriptor_tr_deep_oneleaf_c_20        ... bench:     161,090.85 ns/iter (+/- 796.87)
test benchmarks::parse_descriptor_tr_deep_oneleaf_d_50        ... bench:     389,789.05 ns/iter (+/- 2,249.28)
test benchmarks::parse_descriptor_tr_deep_oneleaf_e_100       ... bench:     791,927.10 ns/iter (+/- 3,733.22)
test benchmarks::parse_descriptor_tr_deep_oneleaf_f_200       ... bench:   1,612,292.60 ns/iter (+/- 4,859.48)
test benchmarks::parse_descriptor_tr_oneleaf_a_1              ... bench:      13,183.04 ns/iter (+/- 74.71)
test benchmarks::parse_descriptor_tr_oneleaf_b_10             ... bench:      79,979.74 ns/iter (+/- 374.92)
test benchmarks::parse_descriptor_tr_oneleaf_c_20             ... bench:     161,018.83 ns/iter (+/- 682.00)
test benchmarks::parse_descriptor_tr_oneleaf_d_50             ... bench:     391,125.15 ns/iter (+/- 1,756.11)
test benchmarks::parse_descriptor_tr_oneleaf_e_100            ... bench:     795,041.20 ns/iter (+/- 4,899.53)
test benchmarks::parse_descriptor_tr_oneleaf_f_200            ... bench:   1,618,193.10 ns/iter (+/- 5,055.56)
test benchmarks::parse_descriptor_tr_oneleaf_g_500            ... bench:   4,150,689.30 ns/iter (+/- 9,215.13)
test benchmarks::parse_descriptor_tr_oneleaf_h_1000           ... bench:   8,541,943.90 ns/iter (+/- 20,539.48)
test benchmarks::parse_descriptor_tr_oneleaf_i_2000           ... bench:  17,330,516.60 ns/iter (+/- 68,171.97)
test benchmarks::parse_descriptor_tr_oneleaf_j_5000           ... bench:  44,243,496.30 ns/iter (+/- 206,636.51)
test benchmarks::parse_descriptor_tr_oneleaf_k_10000          ... bench:  91,907,169.70 ns/iter (+/- 836,489.06)
test benchmarks::parse_expression_balanced_a_0                ... bench:         154.70 ns/iter (+/- 7.52)
test benchmarks::parse_expression_balanced_b_1                ... bench:         518.68 ns/iter (+/- 4.52)
test benchmarks::parse_expression_balanced_c_2                ... bench:         933.45 ns/iter (+/- 9.63)
test benchmarks::parse_expression_balanced_d_5                ... bench:       2,247.19 ns/iter (+/- 13.91)
test benchmarks::parse_expression_balanced_e_10               ... bench:       4,417.51 ns/iter (+/- 31.79)
test benchmarks::parse_expression_balanced_f_20               ... bench:       8,674.86 ns/iter (+/- 883.64)
test benchmarks::parse_expression_balanced_g_50               ... bench:      21,349.21 ns/iter (+/- 144.49)
test benchmarks::parse_expression_balanced_h_100              ... bench:      42,622.33 ns/iter (+/- 237.17)
test benchmarks::parse_expression_balanced_i_200              ... bench:      85,184.04 ns/iter (+/- 943.47)
test benchmarks::parse_expression_balanced_j_500              ... bench:     212,396.08 ns/iter (+/- 957.30)
test benchmarks::parse_expression_balanced_k_1000             ... bench:     425,188.93 ns/iter (+/- 3,195.10)
test benchmarks::parse_expression_balanced_l_2000             ... bench:     850,522.15 ns/iter (+/- 5,239.62)
test benchmarks::parse_expression_balanced_m_5000             ... bench:   2,129,889.80 ns/iter (+/- 18,480.64)
test benchmarks::parse_expression_balanced_n_10000            ... bench:   4,257,929.10 ns/iter (+/- 23,534.12)
test benchmarks::parse_expression_deep_a_0                    ... bench:         143.37 ns/iter (+/- 11.98)
test benchmarks::parse_expression_deep_b_1                    ... bench:         501.64 ns/iter (+/- 2.87)
test benchmarks::parse_expression_deep_c_2                    ... bench:         942.37 ns/iter (+/- 18.88)
test benchmarks::parse_expression_deep_d_5                    ... bench:       2,289.92 ns/iter (+/- 12.66)
test benchmarks::parse_expression_deep_e_10                   ... bench:       4,298.35 ns/iter (+/- 17.28)
test benchmarks::parse_expression_deep_f_20                   ... bench:       8,393.89 ns/iter (+/- 38.80)
test benchmarks::parse_expression_deep_g_50                   ... bench:      20,659.57 ns/iter (+/- 84.18)
test benchmarks::parse_expression_deep_h_100                  ... bench:      40,912.97 ns/iter (+/- 180.31)
test benchmarks::parse_expression_deep_i_200                  ... bench:      82,403.66 ns/iter (+/- 402.95)
test benchmarks::parse_expression_deep_j_300                  ... bench:     122,712.36 ns/iter (+/- 826.80)
test benchmarks::parse_expression_deep_j_400                  ... bench:     163,645.97 ns/iter (+/- 882.22)

You can see a 5-10% improvement across the board, which honestly isn't as great as I was expecting. But this leaves our types and error structures in much better shape so I think it was worth the effort.

sanket1729

ACK 26c6dc3

Nice. 🚀 , no more recursion. Stack overflow bugs.

… of policies and miniscripts non-recursively 26c6dc3a51498dfb641ae2be3db22f8a7a9755d6 miniscript: implement from_tree nonrecursively (Andrew Poelstra) 2c422af5478ea9d1016b007b83a2d13c07be3163 policy: make Concrete::from_tree non-recursive (Andrew Poelstra) 61a89a2655ad04a6bf03329ad730410957635fb9 policy: make Semantic::from_tree non-recursive (Andrew Poelstra) 8a78025160fb8ba6e18c7ccaed728dc54796a743 miniscript: add constructors for all the terminals (Andrew Poelstra) Pull request description: This brings us to the end of my cached "rewrite expression module" branch, by making expression parsing non-recursive. The diff here is actually reasonably short but it's pretty invasive. Hopefully the new fuzz test from the last PR is enough to convince us of correctness. I have some more work to do in two directions: * Breaking up the two `Policy` types into a structs with inner enums (or possibly just a flat tree type), which will allow us to attach metadata to nodes and trees; after this, I will introduce the `ValidationParams` structure and start moving sanity/validation/parsing logic into there as suggested in #723. I have some incomplete commits in this direction but I probably want to redo them with lessons learned from the Taproot work. * Replacing the `TapTree` enum with a non-recursive type, which will let us replace the taproot iteration API with a more natural one (as well as getting some performance benefit). I have more-complete work on this, but I'm getting a bit hamstrung right now handling the difference between `MiniscriptKey` and `ToPublicKey` in this specialization-less language. ACKs for top commit: sanket1729: ACK 26c6dc3a51498dfb641ae2be3db22f8a7a9755d6 Tree-SHA512: b576d779ec1bf495176867e8e35602e1ed8e2a6a9df99fb0058ab437f9e43cf680a0b3bd459c27ba4e21ac58f534b35f7cac6566273c66615d7dc47015243c9a

apoelstra added 4 commits December 17, 2024 19:48

miniscript: add constructors for all the terminals

8a78025

All of the terminals in Miniscript can be constructed infallibly. Add constructors for this so we don't need to go through Miniscript::from_ast and deal with an impossible-to-hit error path.

policy: make Semantic::from_tree non-recursive

61a89a2

policy: make Concrete::from_tree non-recursive

2c422af

sanket1729 approved these changes Dec 19, 2024

View reviewed changes

apoelstra merged commit f3704d1 into rust-bitcoin:master Dec 20, 2024
30 checks passed

apoelstra deleted the 2024-12--expression-5 branch December 20, 2024 13:14

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

expression: implement parsing of policies and miniscripts non-recursively #784

expression: implement parsing of policies and miniscripts non-recursively #784

Uh oh!

apoelstra commented Dec 17, 2024

Uh oh!

apoelstra commented Dec 17, 2024

Uh oh!

sanket1729 left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

expression: implement parsing of policies and miniscripts non-recursively #784

expression: implement parsing of policies and miniscripts non-recursively #784

Uh oh!

Conversation

apoelstra commented Dec 17, 2024

Uh oh!

apoelstra commented Dec 17, 2024

Uh oh!

sanket1729 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants