Skip to content

Commit e15b2d3

Browse files
Amxxfrangioernestognw
authored
Add a 'sortLeaves' options (#29)
Co-authored-by: Francisco <[email protected]> Co-authored-by: Ernesto García <[email protected]>
1 parent 0951d3c commit e15b2d3

File tree

6 files changed

+189
-127
lines changed

6 files changed

+189
-127
lines changed

‎CHANGELOG.md

+4
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,9 @@
11
# Changelog
22

3+
## Unreleased
4+
5+
- Added an option to disable leaf sorting.
6+
37
## 1.0.5
48

59
- Make `processMultiProof` more robust by validating invariants.

‎README.md

+22-1
Original file line numberDiff line numberDiff line change
@@ -130,6 +130,18 @@ bytes32 leaf = keccak256(bytes.concat(keccak256(abi.encode(addr, amount))));
130130

131131
This is an opinionated design that we believe will offer the best out of the box experience for most users. We may introduce options for customization in the future based on user requests.
132132

133+
### Leaf ordering
134+
135+
Each leaf of a merkle tree can be proven individually. The relative ordering of leaves is mostly irrelevant when the only objective is to prove the inclusion of individual leaves in the tree. Proving multiple leaves at once is however a little bit more difficult.
136+
137+
This library proposes a mechanism to prove (and verify) that sets of leaves are included in the tree. These "multiproofs" can also be verified onchain using the implementation available in `@openzeppelin/contracts`. This mechanism requires the leaves to be ordered respective to their position in the tree. For example, if the tree leaves are (in hex form) `[ 0xAA...AA, 0xBB...BB, 0xCC...CC, 0xDD...DD]`, then you'd be able to prove `[0xBB...BB, 0xDD...DD]` as a subset of the leaves, but not `[0xDD...DD, 0xBB...BB]`.
138+
139+
Since this library knows the entire tree, you can generate a multiproof with the requested leaves in any order. The library will re-order them so that they appear inside the proof in the correct order. The `MultiProof` object returned by `tree.getMultiProof(...)` will have the leaves ordered according to their position in the tree, and not in the order in which you provided them.
140+
141+
By default, the library orders the leaves according to their hash when building the tree. This is so that a smart contract can build the hashes of a set of leaves and order them correctly without any knowledge of the tree itself. Said differently, it is simpler for a smart contract to process a multiproof for leaves that it rebuilt itself if the corresponding tree is ordered.
142+
143+
However, some trees are constructed iteratively from unsorted data, causing the leaves to be unsorted as well. For this library to be able to represent such trees, the call to `StandardMerkleTree.of` includes an option to disable sorting. Using that option, the leaves are kept in the order in which they were provided. Note that this option has no effect on your ability to generate and verify proofs and multiproofs in JavaScript, but that it may introduce challenges when verifying multiproofs onchain. We recommend only using it for building a representation of trees that are built (onchain) using an iterative process.
144+
133145
## API & Examples
134146

135147
### `StandardMerkleTree`
@@ -141,14 +153,23 @@ import { StandardMerkleTree } from "@openzeppelin/merkle-tree";
141153
### `StandardMerkleTree.of`
142154

143155
```typescript
144-
const tree = StandardMerkleTree.of([[alice, '100'], [bob, '200']], ['address', 'uint']);
156+
const tree = StandardMerkleTree.of([[alice, '100'], [bob, '200']], ['address', 'uint'], options);
145157
```
146158

147159
Creates a standard merkle tree out of an array of the elements in the tree, along with their types for ABI encoding. For documentation on the syntax of the types, including how to encode structs, refer to the documentation for Ethers.js's [`AbiCoder`](https://docs.ethers.org/v5/api/utils/abi/coder/#AbiCoder-encode).
148160

149161
> **Note**
150162
> Consider reading the array of elements from a CSV file for easy interoperability with spreadsheets or other data processing pipelines.
151163
164+
> **Note**
165+
> By default, leaves are sorted according to their hash. This is done so that multiproof generated by the library can more easily be verified onchain. This can be disabled using the optional third argument. See the [Leaf ordering](#leaf-ordering) section for more details.
166+
167+
#### Options
168+
169+
| Option | Description | Default |
170+
| ------------ | ----------------------------------------------------------------------------------- | ------- |
171+
| `sortLeaves` | Enable or disable sorted leaves. Sorting is strongly recommended for multiproofs. | `true` |
172+
152173
### `StandardMerkleTree.verify`
153174

154175
```typescript

‎src/options.ts

+11
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
// MerkleTree building options
2+
export type MerkleTreeOptions = Partial<{
3+
/** Enable or disable sorted leaves. Sorting is strongly recommended for multiproofs. */
4+
sortLeaves: boolean;
5+
}>;
6+
7+
// Recommended (default) options.
8+
// - leaves are sorted by default to facilitate onchain verification of multiproofs.
9+
export const defaultOptions: Required<MerkleTreeOptions> = {
10+
sortLeaves: true,
11+
};

‎src/standard.test.ts

+134-116
Original file line numberDiff line numberDiff line change
@@ -1,131 +1,149 @@
11
import assert from 'assert/strict';
22
import { keccak256 } from 'ethereum-cryptography/keccak';
33
import { hex } from './bytes';
4+
import { MerkleTreeOptions } from './options';
45
import { StandardMerkleTree } from './standard';
56

67
const zeroBytes = new Uint8Array(32);
78
const zero = hex(zeroBytes);
89

9-
const characters = (s: string) => {
10+
const makeTree = (s: string, opts: MerkleTreeOptions = {}) => {
1011
const l = s.split('').map(c => [c]);
11-
const t = StandardMerkleTree.of(l, ['string']);
12+
const t = StandardMerkleTree.of(l, ['string'], opts);
1213
return { l, t };
1314
}
1415

1516
describe('standard merkle tree', () => {
16-
it('generates valid single proofs for all leaves', () => {
17-
const { t } = characters('abcdef');
18-
t.validate();
19-
});
20-
21-
it('generates valid single proofs for all leaves', () => {
22-
const { t } = characters('abcdef');
23-
24-
for (const [id, leaf] of t.entries()) {
25-
const proof1 = t.getProof(id);
26-
const proof2 = t.getProof(leaf);
27-
28-
assert.deepEqual(proof1, proof2);
29-
30-
assert(t.verify(id, proof1));
31-
assert(t.verify(leaf, proof1));
32-
assert(StandardMerkleTree.verify(t.root, ['string'], leaf, proof1));
33-
}
34-
});
35-
36-
it('rejects invalid proofs', () => {
37-
const { t } = characters('abcdef');
38-
const { t: otherTree } = characters('abc');
39-
40-
const leaf = ['a'];
41-
const invalidProof = otherTree.getProof(leaf);
42-
43-
assert(!t.verify(leaf, invalidProof));
44-
assert(!StandardMerkleTree.verify(t.root, ['string'], leaf, invalidProof));
45-
});
46-
47-
it('generates valid multiproofs', () => {
48-
const { t, l } = characters('abcdef');
49-
50-
for (const ids of [[], [0, 1], [0, 1, 5], [1, 3, 4, 5], [0, 2, 4, 5], [0, 1, 2, 3, 4, 5]]) {
51-
const proof1 = t.getMultiProof(ids);
52-
const proof2 = t.getMultiProof(ids.map(i => l[i]!));
53-
54-
assert.deepEqual(proof1, proof2);
55-
56-
assert(t.verifyMultiProof(proof1));
57-
assert(StandardMerkleTree.verifyMultiProof(t.root, ['string'], proof1));
58-
}
59-
});
60-
61-
it('rejects invalid multiproofs', () => {
62-
const { t } = characters('abcdef');
63-
const { t: otherTree } = characters('abc');
64-
65-
const leaves = [['a'], ['b'], ['c']];
66-
const multiProof = otherTree.getMultiProof(leaves);
67-
68-
assert(!t.verifyMultiProof(multiProof));
69-
assert(!StandardMerkleTree.verifyMultiProof(t.root, ['string'], multiProof));
70-
});
71-
72-
it('renders tree representation', () => {
73-
const { t } = characters('abc');
74-
75-
const expected = `\
76-
0) f2129b5a697531ef818f644564a6552b35c549722385bc52aa7fe46c0b5f46b1
77-
├─ 1) fa914d99a18dc32d9725b3ef1c50426deb40ec8d0885dac8edcc5bfd6d030016
78-
│ ├─ 3) 9c15a6a0eaeed500fd9eed4cbeab71f797cefcc67bfd46683e4d2e6ff7f06d1c
79-
│ └─ 4) 19ba6c6333e0e9a15bf67523e0676e2f23eb8e574092552d5e888c64a4bb3681
80-
└─ 2) 9cf5a63718145ba968a01c1d557020181c5b252f665cf7386d370eddb176517b`;
81-
82-
assert.equal(t.render(), expected);
83-
});
84-
85-
it('dump and load', () => {
86-
const { t } = characters('abcdef');
87-
const t2 = StandardMerkleTree.load(t.dump());
88-
89-
t2.validate();
90-
assert.deepEqual(t2, t);
91-
});
92-
93-
it('reject out of bounds value index', () => {
94-
const { t } = characters('a');
95-
assert.throws(
96-
() => t.getProof(1),
97-
/^Error: Index out of bounds$/,
98-
);
99-
});
100-
101-
it('reject unrecognized tree dump', () => {
102-
assert.throws(
103-
() => StandardMerkleTree.load({ format: 'nonstandard' } as any),
104-
/^Error: Unknown format 'nonstandard'$/,
105-
);
106-
});
107-
108-
it('reject malformed tree dump', () => {
109-
const t1 = StandardMerkleTree.load({
110-
format: 'standard-v1',
111-
tree: [zero],
112-
values: [{ value: ['0'], treeIndex: 0 }],
113-
leafEncoding: ['uint256'],
17+
for (const opts of [
18+
{},
19+
{ sortLeaves: true },
20+
{ sortLeaves: false },
21+
]) {
22+
describe(`with options '${JSON.stringify(opts)}'`, () => {
23+
const { l: leaves, t: tree } = makeTree('abcdef', opts);
24+
const { l: otherLeaves, t: otherTree } = makeTree('abc', opts);
25+
26+
it('generates valid single proofs for all leaves', () => {
27+
tree.validate();
28+
});
29+
30+
it('generates valid single proofs for all leaves', () => {
31+
for (const [id, leaf] of tree.entries()) {
32+
const proof1 = tree.getProof(id);
33+
const proof2 = tree.getProof(leaf);
34+
35+
assert.deepEqual(proof1, proof2);
36+
37+
assert(tree.verify(id, proof1));
38+
assert(tree.verify(leaf, proof1));
39+
assert(StandardMerkleTree.verify(tree.root, ['string'], leaf, proof1));
40+
}
41+
});
42+
43+
it('rejects invalid proofs', () => {
44+
const leaf = ['a'];
45+
const invalidProof = otherTree.getProof(leaf);
46+
47+
assert(!tree.verify(leaf, invalidProof));
48+
assert(!StandardMerkleTree.verify(tree.root, ['string'], leaf, invalidProof));
49+
});
50+
51+
it('generates valid multiproofs', () => {
52+
for (const ids of [[], [0, 1], [0, 1, 5], [1, 3, 4, 5], [0, 2, 4, 5], [0, 1, 2, 3, 4, 5], [4, 1, 5, 0, 2]]) {
53+
const proof1 = tree.getMultiProof(ids);
54+
const proof2 = tree.getMultiProof(ids.map(i => leaves[i]!));
55+
56+
assert.deepEqual(proof1, proof2);
57+
58+
assert(tree.verifyMultiProof(proof1));
59+
assert(StandardMerkleTree.verifyMultiProof(tree.root, ['string'], proof1));
60+
}
61+
});
62+
63+
it('rejects invalid multiproofs', () => {
64+
const multiProof = otherTree.getMultiProof([['a'], ['b'], ['c']]);
65+
66+
assert(!tree.verifyMultiProof(multiProof));
67+
assert(!StandardMerkleTree.verifyMultiProof(tree.root, ['string'], multiProof));
68+
});
69+
70+
it('renders tree representation', () => {
71+
assert.equal(
72+
tree.render(),
73+
opts.sortLeaves == false
74+
? [
75+
"0) 23be0977360f08bb0bd7f709a7d543d2cd779c79c66d74e0441919871647de2b",
76+
"├─ 1) 8f7234e8cfe39c08ca84a3a3e3274f574af26fd15165fe29e09cbab742daccd9",
77+
"│ ├─ 3) 03707d7802a71ca56a8ad8028da98c4f1dbec55b31b4a25d536b5309cc20eda9",
78+
"│ │ ├─ 7) eba909cf4bb90c6922771d7f126ad0fd11dfde93f3937a196274e1ac20fd2f5b",
79+
"│ │ └─ 8) 9cf5a63718145ba968a01c1d557020181c5b252f665cf7386d370eddb176517b",
80+
"│ └─ 4) fa914d99a18dc32d9725b3ef1c50426deb40ec8d0885dac8edcc5bfd6d030016",
81+
"│ ├─ 9) 19ba6c6333e0e9a15bf67523e0676e2f23eb8e574092552d5e888c64a4bb3681",
82+
"│ └─ 10) 9c15a6a0eaeed500fd9eed4cbeab71f797cefcc67bfd46683e4d2e6ff7f06d1c",
83+
"└─ 2) 7b0c6cd04b82bfc0e250030a5d2690c52585e0cc6a4f3bc7909d7723b0236ece",
84+
" ├─ 5) c62a8cfa41edc0ef6f6ae27a2985b7d39c7fea770787d7e104696c6e81f64848",
85+
" └─ 6) 9a4f64e953595df82d1b4f570d34c4f4f0cfaf729a61e9d60e83e579e1aa283e",
86+
].join("\n")
87+
: [
88+
"0) 6deb52b5da8fd108f79fab00341f38d2587896634c646ee52e49f845680a70c8",
89+
"├─ 1) 52426e0f1f65ff7e209a13b8c29cffe82e3acaf3dad0a9b9088f3b9a61a929c3",
90+
"│ ├─ 3) 8076923e76cf01a7c048400a2304c9a9c23bbbdac3a98ea3946340fdafbba34f",
91+
"│ │ ├─ 7) 9cf5a63718145ba968a01c1d557020181c5b252f665cf7386d370eddb176517b",
92+
"│ │ └─ 8) 9c15a6a0eaeed500fd9eed4cbeab71f797cefcc67bfd46683e4d2e6ff7f06d1c",
93+
"│ └─ 4) 965b92c6cf08303cc4feb7f3e0819c436c2cec17c6f0688a6af139c9a368707c",
94+
"│ ├─ 9) 9a4f64e953595df82d1b4f570d34c4f4f0cfaf729a61e9d60e83e579e1aa283e",
95+
"│ └─ 10) 19ba6c6333e0e9a15bf67523e0676e2f23eb8e574092552d5e888c64a4bb3681",
96+
"└─ 2) fd3cf45654e88d1cc5d663578c82c76f4b5e3826bacaa1216441443504538f51",
97+
" ├─ 5) eba909cf4bb90c6922771d7f126ad0fd11dfde93f3937a196274e1ac20fd2f5b",
98+
" └─ 6) c62a8cfa41edc0ef6f6ae27a2985b7d39c7fea770787d7e104696c6e81f64848",
99+
].join("\n"),
100+
);
101+
});
102+
103+
it('dump and load', () => {
104+
const recoveredTree = StandardMerkleTree.load(tree.dump());
105+
106+
recoveredTree.validate();
107+
assert.deepEqual(tree, recoveredTree);
108+
});
109+
110+
it('reject out of bounds value index', () => {
111+
assert.throws(
112+
() => tree.getProof(leaves.length),
113+
/^Error: Index out of bounds$/,
114+
);
115+
});
116+
117+
it('reject unrecognized tree dump', () => {
118+
assert.throws(
119+
() => StandardMerkleTree.load({ format: 'nonstandard' } as any),
120+
/^Error: Unknown format 'nonstandard'$/,
121+
);
122+
});
123+
124+
it('reject malformed tree dump', () => {
125+
const loadedTree1 = StandardMerkleTree.load({
126+
format: 'standard-v1',
127+
tree: [zero],
128+
values: [{ value: ['0'], treeIndex: 0 }],
129+
leafEncoding: ['uint256'],
130+
});
131+
assert.throws(
132+
() => loadedTree1.getProof(0),
133+
/^Error: Merkle tree does not contain the expected value$/,
134+
);
135+
136+
const loadedTree2 = StandardMerkleTree.load({
137+
format: 'standard-v1',
138+
tree: [zero, zero, hex(keccak256(keccak256(zeroBytes)))],
139+
values: [{ value: ['0'], treeIndex: 2 }],
140+
leafEncoding: ['uint256'],
141+
});
142+
assert.throws(
143+
() => loadedTree2.getProof(0),
144+
/^Error: Unable to prove value$/,
145+
);
146+
});
114147
});
115-
assert.throws(
116-
() => t1.getProof(0),
117-
/^Error: Merkle tree does not contain the expected value$/,
118-
);
119-
120-
const t2 = StandardMerkleTree.load({
121-
format: 'standard-v1',
122-
tree: [zero, zero, hex(keccak256(keccak256(zeroBytes)))],
123-
values: [{ value: ['0'], treeIndex: 2 }],
124-
leafEncoding: ['uint256'],
125-
});
126-
assert.throws(
127-
() => t2.getProof(0),
128-
/^Error: Unable to prove value$/,
129-
);
130-
});
148+
}
131149
});

‎src/standard.ts

+10-10
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,10 @@
1-
import { keccak256 } from 'ethereum-cryptography/keccak';
21
import { equalsBytes, hexToBytes } from 'ethereum-cryptography/utils';
3-
import { defaultAbiCoder } from '@ethersproject/abi';
42
import { Bytes, compareBytes, hex } from './bytes';
53
import { getProof, isValidMerkleTree, makeMerkleTree, processProof, renderMerkleTree, MultiProof, getMultiProof, processMultiProof } from './core';
4+
import { MerkleTreeOptions, defaultOptions } from './options';
65
import { checkBounds } from './utils/check-bounds';
76
import { throwError } from './utils/throw-error';
8-
9-
function standardLeafHash<T extends any[]>(value: T, types: string[]): Bytes {
10-
return keccak256(keccak256(hexToBytes(defaultAbiCoder.encode(types, value))));
11-
}
7+
import { standardLeafHash } from './utils/standard-leaf-hash';
128

139
interface StandardMerkleTreeData<T extends any[]> {
1410
format: 'standard-v1';
@@ -35,10 +31,14 @@ export class StandardMerkleTree<T extends any[]> {
3531
]));
3632
}
3733

38-
static of<T extends any[]>(values: T[], leafEncoding: string[]) {
39-
const hashedValues = values
40-
.map((value, valueIndex) => ({ value, valueIndex, hash: standardLeafHash(value, leafEncoding) }))
41-
.sort((a, b) => compareBytes(a.hash, b.hash));
34+
static of<T extends any[]>(values: T[], leafEncoding: string[], options: MerkleTreeOptions = {}) {
35+
const sortLeaves = options.sortLeaves ?? defaultOptions.sortLeaves;
36+
37+
const hashedValues = values.map((value, valueIndex) => ({ value, valueIndex, hash: standardLeafHash(value, leafEncoding) }));
38+
39+
if (sortLeaves) {
40+
hashedValues.sort((a, b) => compareBytes(a.hash, b.hash));
41+
}
4242

4343
const tree = makeMerkleTree(hashedValues.map(v => v.hash));
4444

‎src/utils/standard-leaf-hash.ts

+8
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
import { keccak256 } from 'ethereum-cryptography/keccak';
2+
import { hexToBytes } from 'ethereum-cryptography/utils';
3+
import { defaultAbiCoder } from '@ethersproject/abi';
4+
import { Bytes } from '../bytes';
5+
6+
export function standardLeafHash<T extends any[]>(value: T, types: string[]): Bytes {
7+
return keccak256(keccak256(hexToBytes(defaultAbiCoder.encode(types, value))));
8+
}

0 commit comments

Comments
 (0)