Skip to content

Commit 002f1b2

Browse files
PastaPastaPastaPastaPastaPasta
committed
Merge #6969: feat: add evodb verify and repair RPC commands
f8be956 refactor: rename second BuildNewListFromBlock variant to RebuildListFromBlock (UdjinM6) f3ee1d7 refactor: move verification progress log out of VerifySnapshotPair (UdjinM6) f482f01 fix: drop redundant thread safety annotations (UdjinM6) 6df3a7c docs: add release notes for evodb verify/repair RPCs (UdjinM6) 7decd15 docs: add comprehensive evodb verify/repair documentation (UdjinM6) 17b556a feat: add evodb verify and repair RPC commands (UdjinM6) 5379fce refactor: add CDeterministicMNList::IsEqual method (UdjinM6) 496ed89 refactor: add BuildNewListFromBlock overload accepting explicit prevList (UdjinM6) Pull request description: ## Issue being fixed or feature implemented #6961 should fix the issue for new nodes migrating to v23. This PR aims to provide tools for nodes that updated before the fix implemented in #6961. ## What was done? Preparatory Refactoring (2 commits) 1. BuildNewListFromBlock overload - Added method overload that accepts explicit prevList parameter instead of loading from database. Needed for repair to rebuild from trusted snapshots. 2. CDeterministicMNList::IsEqual - Added equality comparison method to verify recalculated diffs produce correct results. Main Feature (1 commit) 3. evodb verify and repair RPCs - Core implementation: - Two new hidden RPC commands for diagnosing/repairing corrupted evodb diffs - evodb verify - Read-only verification between snapshots (every 576 blocks) - evodb repair - Recalculates corrupted diffs from blockchain data - Fail-fast on critical errors, efficient 16MB batched writes, cache clearing Documentation (2 commits) 4. Comprehensive docs - Full technical documentation in doc/evodb-verify-repair.md 5. Release notes - Short user-facing notes in doc/release-notes-6969.md Key Points - Purpose: Repair corrupted evodb diffs without full reindex (when possible) - Use cases: After crashes, disk corruption, or suspected database issues - Limitations: Can't repair missing snapshots (needs reindex), requires unpruned blocks - Performance: I/O intensive, logs progress every 100 snapshot pairs - Safety: Verifies repairs before committing, clears caches, clear error messages ## How Has This Been Tested? 1. Sync on testnet/mainet with v22.1.3 (save evodb from datadir for repeated experiments) and stop the node 2. Start v23 (with `--noconnect` to avoid altering blocks and chainstate), wait for migration to finish and stop the node 3. Start a node with this patch and run: 1. `evodb verify`, should see a bunch of errors in `verificationErrors` 2. `evodb repair`, should see same errors in `verificationErrors` and none in `repairErrors` (can specify start/stop params from errors in `verificationErrors` to speed things up a bit, `verificationErrors` should look accordingly) 3. `evodb verify` or `evodb repair` again, should see no errors now ## Breaking Changes n/a ## Checklist: - [ ] I have performed a self-review of my own code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have added or updated relevant unit/integration/functional/e2e tests - [ ] I have made corresponding changes to the documentation - [ ] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_ ACKs for top commit: PastaPastaPasta: utACK f8be956 Tree-SHA512: 0cddb7f9fb3ae5519ae8cdb868aafbaeace35ca9f5fa7319d7ddbee3466e7765a21a8bae402983e3fc8f0a2ffab32c8bc60149af65bc2522143ca939b081c605
2 parents f047d8b + f8be956 commit 002f1b2

File tree

7 files changed

+785
-8
lines changed

7 files changed

+785
-8
lines changed

‎doc/evodb-verify-repair.md‎

Lines changed: 160 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,160 @@
1+
# EvoDb Verify and Repair RPC Commands
2+
3+
### `evodb verify`
4+
5+
Verifies the integrity of evodb diff records between snapshots stored every 576 blocks.
6+
7+
**Syntax:**
8+
```
9+
evodb verify [startBlock] [stopBlock]
10+
```
11+
12+
**Parameters:**
13+
- `startBlock` (optional): Starting block height. Defaults to DIP0003 activation height.
14+
- `stopBlock` (optional): Ending block height. Defaults to current chain tip.
15+
16+
**Returns:**
17+
```json
18+
{
19+
"startHeight": n, // Actual starting height (may be clamped to DIP0003)
20+
"stopHeight": n, // Ending block height
21+
"diffsRecalculated": 0, // Always 0 for verify mode
22+
"snapshotsVerified": n, // Number of snapshot pairs that passed verification
23+
"verificationErrors": [ // Array of errors (empty if verification passed)
24+
"error message",
25+
...
26+
]
27+
}
28+
```
29+
30+
**Description:**
31+
32+
This is a **read-only** operation that checks whether applying the stored diffs between consecutive snapshots produces the expected results. It does not modify the database.
33+
34+
The command processes snapshot pairs (snapshots are stored every 576 blocks) and applies all diffs between them, verifying that the result matches the target snapshot. If all verification passes, `verificationErrors` will be empty.
35+
36+
**Use cases:**
37+
- Diagnose suspected evodb corruption
38+
- Verify database integrity after hardware issues
39+
- Confirm successful repair operations
40+
41+
**Example:**
42+
```bash
43+
# Verify entire chain
44+
dash-cli evodb verify
45+
46+
# Verify specific range
47+
dash-cli evodb verify 1000 10000
48+
```
49+
50+
---
51+
52+
### `evodb repair`
53+
54+
Repairs corrupted evodb diff records by recalculating them from blockchain data.
55+
56+
**Syntax:**
57+
```
58+
evodb repair [startBlock] [stopBlock]
59+
```
60+
61+
**Parameters:**
62+
- `startBlock` (optional): Starting block height. Defaults to DIP0003 activation height.
63+
- `stopBlock` (optional): Ending block height. Defaults to current chain tip.
64+
65+
**Returns:**
66+
```json
67+
{
68+
"startHeight": n, // Actual starting height (may be clamped to DIP0003)
69+
"stopHeight": n, // Ending block height
70+
"diffsRecalculated": n, // Number of diffs successfully recalculated
71+
"snapshotsVerified": n, // Number of snapshot pairs that passed verification
72+
"verificationErrors": [ // Errors during verification phase
73+
"error message",
74+
...
75+
],
76+
"repairErrors": [ // Critical errors during repair phase
77+
"error message", // Non-empty means full reindex required
78+
...
79+
]
80+
}
81+
```
82+
83+
**Description:**
84+
85+
This command first runs verification on all snapshot pairs in the specified range. For any pairs that fail verification, it recalculates the diffs from actual blockchain data and writes the corrected diffs to the database.
86+
87+
The repair process:
88+
1. **Verification phase**: Checks all snapshot pairs for corruption
89+
2. **Repair phase**: For failed pairs, recalculates diffs from blockchain blocks
90+
3. **Database update**: Writes repaired diffs in efficient 16MB batches
91+
4. **Cache clearing**: Clears both diff and list caches to prevent serving stale data
92+
93+
**Important notes:**
94+
- Requires all blockchain data to be available (blocks must not be pruned in the repair range)
95+
- If repair encounters critical errors (block read failures, missing snapshots), a full reindex is required
96+
- Critical errors are prefixed with "CRITICAL:" in error messages
97+
- Successfully repaired diffs are verified before being committed to the database
98+
99+
**Use cases:**
100+
- Repair corrupted evodb after unclean shutdown
101+
- Fix database inconsistencies after hardware failures
102+
- Recover from disk corruption without full reindex (when possible)
103+
104+
**Example:**
105+
```bash
106+
# Repair entire chain
107+
dash-cli evodb repair
108+
109+
# Repair specific range
110+
dash-cli evodb repair 1000 10000
111+
```
112+
113+
---
114+
115+
## When to Use These Commands
116+
117+
### Use `evodb verify` when:
118+
- You suspect evodb corruption but want to diagnose before taking action
119+
- Verifying integrity after hardware issues or crashes
120+
- Confirming successful repairs
121+
122+
### Use `evodb repair` when:
123+
- `evodb verify` reports verification errors
124+
- You experience masternode list inconsistencies
125+
- After unclean shutdown or disk errors
126+
- As an alternative to full reindex when snapshots are intact
127+
128+
### Full reindex required when:
129+
- Repair reports errors prefixed with "CRITICAL:"
130+
- Snapshots are missing or corrupted (cannot be repaired by this tool)
131+
- Block data is unavailable (pruned nodes in repair range)
132+
133+
---
134+
135+
## Technical Details
136+
137+
### Verification Process
138+
- Processes snapshot pairs sequentially (snapshots stored every 576 blocks)
139+
- Applies all stored diffs between snapshots
140+
- Verifies result matches target snapshot using `CDeterministicMNList::IsEqual`
141+
- Reports all errors without stopping early
142+
143+
### Repair Process
144+
- Reads actual blockchain blocks from disk
145+
- Processes special transactions to rebuild masternode lists
146+
- Uses dummy coins view (avoids UTXO lookups for historical blocks)
147+
- Calculates correct diffs and verifies before committing
148+
- Fails fast on critical errors (missing blocks, missing snapshots)
149+
150+
### Performance Considerations
151+
- Repair is I/O intensive (reads blockchain blocks, writes database)
152+
- Progress logged every 100 snapshot pairs
153+
- Database writes batched in 16MB chunks for efficiency
154+
- Caches cleared after repair to ensure consistency
155+
156+
### Implementation Notes
157+
- Both commands require `::cs_main` lock
158+
- Special handling for initial DIP0003 snapshot (may not exist in older databases)
159+
- Only diffs can be repaired; missing snapshots require full reindex
160+
- Repair verification must pass before diffs are committed to database

‎doc/release-notes-6969.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
RPC changes
2+
-----------
3+
4+
- Two new hidden RPC commands have been added for diagnosing and repairing corrupted evodb diff records:
5+
- `evodb verify` - Verifies the integrity of evodb diff records between snapshots (read-only operation). Returns verification errors if any corruption is detected.
6+
- `evodb repair` - Repairs corrupted evodb diff records by recalculating them from blockchain data. Automatically verifies repairs before committing to database. Reports critical errors that require full reindex.

0 commit comments

Comments
 (0)