diff --git a/doc/evodb-verify-repair.md b/doc/evodb-verify-repair.md new file mode 100644 index 0000000000000..5ebb0edf63ec5 --- /dev/null +++ b/doc/evodb-verify-repair.md @@ -0,0 +1,160 @@ +# EvoDb Verify and Repair RPC Commands + +### `evodb verify` + +Verifies the integrity of evodb diff records between snapshots stored every 576 blocks. + +**Syntax:** +``` +evodb verify [startBlock] [stopBlock] +``` + +**Parameters:** +- `startBlock` (optional): Starting block height. Defaults to DIP0003 activation height. +- `stopBlock` (optional): Ending block height. Defaults to current chain tip. + +**Returns:** +```json +{ + "startHeight": n, // Actual starting height (may be clamped to DIP0003) + "stopHeight": n, // Ending block height + "diffsRecalculated": 0, // Always 0 for verify mode + "snapshotsVerified": n, // Number of snapshot pairs that passed verification + "verificationErrors": [ // Array of errors (empty if verification passed) + "error message", + ... + ] +} +``` + +**Description:** + +This is a **read-only** operation that checks whether applying the stored diffs between consecutive snapshots produces the expected results. It does not modify the database. + +The command processes snapshot pairs (snapshots are stored every 576 blocks) and applies all diffs between them, verifying that the result matches the target snapshot. If all verification passes, `verificationErrors` will be empty. + +**Use cases:** +- Diagnose suspected evodb corruption +- Verify database integrity after hardware issues +- Confirm successful repair operations + +**Example:** +```bash +# Verify entire chain +dash-cli evodb verify + +# Verify specific range +dash-cli evodb verify 1000 10000 +``` + +--- + +### `evodb repair` + +Repairs corrupted evodb diff records by recalculating them from blockchain data. + +**Syntax:** +``` +evodb repair [startBlock] [stopBlock] +``` + +**Parameters:** +- `startBlock` (optional): Starting block height. Defaults to DIP0003 activation height. +- `stopBlock` (optional): Ending block height. Defaults to current chain tip. + +**Returns:** +```json +{ + "startHeight": n, // Actual starting height (may be clamped to DIP0003) + "stopHeight": n, // Ending block height + "diffsRecalculated": n, // Number of diffs successfully recalculated + "snapshotsVerified": n, // Number of snapshot pairs that passed verification + "verificationErrors": [ // Errors during verification phase + "error message", + ... + ], + "repairErrors": [ // Critical errors during repair phase + "error message", // Non-empty means full reindex required + ... + ] +} +``` + +**Description:** + +This command first runs verification on all snapshot pairs in the specified range. For any pairs that fail verification, it recalculates the diffs from actual blockchain data and writes the corrected diffs to the database. + +The repair process: +1. **Verification phase**: Checks all snapshot pairs for corruption +2. **Repair phase**: For failed pairs, recalculates diffs from blockchain blocks +3. **Database update**: Writes repaired diffs in efficient 16MB batches +4. **Cache clearing**: Clears both diff and list caches to prevent serving stale data + +**Important notes:** +- Requires all blockchain data to be available (blocks must not be pruned in the repair range) +- If repair encounters critical errors (block read failures, missing snapshots), a full reindex is required +- Critical errors are prefixed with "CRITICAL:" in error messages +- Successfully repaired diffs are verified before being committed to the database + +**Use cases:** +- Repair corrupted evodb after unclean shutdown +- Fix database inconsistencies after hardware failures +- Recover from disk corruption without full reindex (when possible) + +**Example:** +```bash +# Repair entire chain +dash-cli evodb repair + +# Repair specific range +dash-cli evodb repair 1000 10000 +``` + +--- + +## When to Use These Commands + +### Use `evodb verify` when: +- You suspect evodb corruption but want to diagnose before taking action +- Verifying integrity after hardware issues or crashes +- Confirming successful repairs + +### Use `evodb repair` when: +- `evodb verify` reports verification errors +- You experience masternode list inconsistencies +- After unclean shutdown or disk errors +- As an alternative to full reindex when snapshots are intact + +### Full reindex required when: +- Repair reports errors prefixed with "CRITICAL:" +- Snapshots are missing or corrupted (cannot be repaired by this tool) +- Block data is unavailable (pruned nodes in repair range) + +--- + +## Technical Details + +### Verification Process +- Processes snapshot pairs sequentially (snapshots stored every 576 blocks) +- Applies all stored diffs between snapshots +- Verifies result matches target snapshot using `CDeterministicMNList::IsEqual` +- Reports all errors without stopping early + +### Repair Process +- Reads actual blockchain blocks from disk +- Processes special transactions to rebuild masternode lists +- Uses dummy coins view (avoids UTXO lookups for historical blocks) +- Calculates correct diffs and verifies before committing +- Fails fast on critical errors (missing blocks, missing snapshots) + +### Performance Considerations +- Repair is I/O intensive (reads blockchain blocks, writes database) +- Progress logged every 100 snapshot pairs +- Database writes batched in 16MB chunks for efficiency +- Caches cleared after repair to ensure consistency + +### Implementation Notes +- Both commands require `::cs_main` lock +- Special handling for initial DIP0003 snapshot (may not exist in older databases) +- Only diffs can be repaired; missing snapshots require full reindex +- Repair verification must pass before diffs are committed to database diff --git a/doc/release-notes-6969.md b/doc/release-notes-6969.md new file mode 100644 index 0000000000000..ee3a494cd4fb6 --- /dev/null +++ b/doc/release-notes-6969.md @@ -0,0 +1,6 @@ +RPC changes +----------- + +- Two new hidden RPC commands have been added for diagnosing and repairing corrupted evodb diff records: + - `evodb verify` - Verifies the integrity of evodb diff records between snapshots (read-only operation). Returns verification errors if any corruption is detected. + - `evodb repair` - Repairs corrupted evodb diff records by recalculating them from blockchain data. Automatically verifies repairs before committing to database. Reports critical errors that require full reindex. diff --git a/src/evo/deterministicmns.cpp b/src/evo/deterministicmns.cpp index d8140aa0db900..a0dff77e8cf6d 100644 --- a/src/evo/deterministicmns.cpp +++ b/src/evo/deterministicmns.cpp @@ -25,6 +25,7 @@ #include #include +#include #include #include @@ -1578,3 +1579,336 @@ bool CDeterministicMNManager::MigrateLegacyDiffs(const CBlockIndex* const tip_in return true; } + +CDeterministicMNManager::RecalcDiffsResult CDeterministicMNManager::RecalculateAndRepairDiffs( + const CBlockIndex* start_index, const CBlockIndex* stop_index, ChainstateManager& chainman, + BuildListFromBlockFunc build_list_func, bool repair) +{ + AssertLockHeld(::cs_main); + + RecalcDiffsResult result; + result.start_height = start_index->nHeight; + result.stop_height = stop_index->nHeight; + + const auto& consensus_params = Params().GetConsensus(); + + // Clamp start height to DIP0003 activation (no snapshots/diffs exist before this) + if (start_index->nHeight < consensus_params.DIP0003Height) { + start_index = stop_index->GetAncestor(consensus_params.DIP0003Height); + if (!start_index) { + result.verification_errors.push_back(strprintf("Stop height %d is below DIP0003 activation height %d", + stop_index->nHeight, consensus_params.DIP0003Height)); + return result; + } + LogPrintf("CDeterministicMNManager::%s -- Clamped start height from %d to DIP0003 activation height %d\n", + __func__, result.start_height, consensus_params.DIP0003Height); + // Update result to reflect the clamped start height + result.start_height = start_index->nHeight; + } + + // Collect all snapshot blocks in the range + std::vector snapshot_blocks = CollectSnapshotBlocks(start_index, stop_index, consensus_params); + + if (snapshot_blocks.empty()) { + result.verification_errors.push_back("Could not find starting snapshot"); + return result; + } + + if (snapshot_blocks.size() < 2) { + result.verification_errors.push_back(strprintf("Need at least 2 snapshots, found %d", snapshot_blocks.size())); + return result; + } + + LogPrintf("CDeterministicMNManager::%s -- Processing %d snapshot pairs between heights %d and %d\n", __func__, + snapshot_blocks.size() - 1, result.start_height, result.stop_height); + + // Storage for recalculated diffs if we plan to repair + std::vector> recalculated_diffs; + + // Process each pair of consecutive snapshots + for (size_t i = 0; i < snapshot_blocks.size() - 1; ++i) { + const CBlockIndex* from_index = snapshot_blocks[i]; + const CBlockIndex* to_index = snapshot_blocks[i + 1]; + + // Load the snapshots from disk + CDeterministicMNList from_snapshot; + CDeterministicMNList to_snapshot; + + bool has_from_snapshot = m_evoDb.Read(std::make_pair(DB_LIST_SNAPSHOT, from_index->GetBlockHash()), from_snapshot); + bool has_to_snapshot = m_evoDb.Read(std::make_pair(DB_LIST_SNAPSHOT, to_index->GetBlockHash()), to_snapshot); + + // Handle missing snapshots + if (!has_from_snapshot) { + // The initial snapshot at DIP0003 activation might not exist in the database on nodes + // that synced before the fix to explicitly write it. This is the only acceptable case. + if (from_index->nHeight == consensus_params.DIP0003Height) { + // Create an empty initial snapshot (matching what GetListForBlockInternal does) + from_snapshot = CDeterministicMNList(from_index->GetBlockHash(), from_index->nHeight, 0); + LogPrintf("CDeterministicMNManager::%s -- Using empty initial snapshot at DIP0003 height %d\n", + __func__, from_index->nHeight); + } else { + // Any other missing snapshot is critical corruption beyond our repair capability + result.verification_errors.push_back(strprintf("CRITICAL: Snapshot missing at height %d. " + "This cannot be repaired by this tool - full reindex required.", from_index->nHeight)); + return result; + } + } + + if (!has_to_snapshot) { + // Missing target snapshot is always critical - we cannot repair snapshots, only diffs + result.verification_errors.push_back(strprintf("CRITICAL: Snapshot missing at height %d. " + "This cannot be repaired by this tool - full reindex required.", to_index->nHeight)); + return result; + } + + // Log progress periodically (every 100 snapshot pairs) to avoid spam + if (i % 100 == 0) { + LogPrintf("CDeterministicMNManager::%s -- Progress: verifying snapshot pair %d/%d (heights %d-%d)\n", + __func__, i + 1, snapshot_blocks.size() - 1, from_index->nHeight, to_index->nHeight); + } + + // Verify this snapshot pair + bool is_snapshot_pair_valid = VerifySnapshotPair(from_index, to_index, from_snapshot, to_snapshot, result); + + // If repair mode is enabled and verification failed, recalculate diffs from blockchain + if (repair && !is_snapshot_pair_valid) { + auto temp_diffs = RepairSnapshotPair(from_index, to_index, from_snapshot, to_snapshot, build_list_func, result); + if (temp_diffs.empty()) { + // RepairSnapshotPair failed - this is a critical error, cannot continue + return result; + } + // Only commit diffs if recalculation verification passed + recalculated_diffs.insert(recalculated_diffs.end(), temp_diffs.begin(), temp_diffs.end()); + result.diffs_recalculated += temp_diffs.size(); + } + } + + // Write repaired diffs to database + if (repair) { + WriteRepairedDiffs(recalculated_diffs, result); + } + + return result; +} + +std::vector CDeterministicMNManager::CollectSnapshotBlocks( + const CBlockIndex* start_index, const CBlockIndex* stop_index, const Consensus::Params& consensus_params) +{ + AssertLockHeld(::cs_main); + + std::vector snapshot_blocks; + + // Add the starting snapshot (find the snapshot at or before start) + // Walk backwards to find a snapshot block (divisible by DISK_SNAPSHOT_PERIOD) + // or the initial snapshot at DIP0003 activation height + const CBlockIndex* snapshot_start_index = start_index; + while (snapshot_start_index && snapshot_start_index->nHeight > consensus_params.DIP0003Height && + (snapshot_start_index->nHeight % DISK_SNAPSHOT_PERIOD) != 0) { + snapshot_start_index = snapshot_start_index->pprev; + } + + if (!snapshot_start_index) { + return snapshot_blocks; // Empty vector indicates error + } + + // Collect all snapshot blocks up to and including the stop block + snapshot_blocks.push_back(snapshot_start_index); + + // Find all subsequent snapshot heights + int current_snapshot_height = snapshot_start_index->nHeight; + while (true) { + // Calculate next snapshot height + int next_snapshot_height; + if (current_snapshot_height == consensus_params.DIP0003Height) { + // If we're at DIP0003 activation (initial snapshot), next is at first regular interval + next_snapshot_height = ((consensus_params.DIP0003Height / DISK_SNAPSHOT_PERIOD) + 1) * DISK_SNAPSHOT_PERIOD; + } else { + // Otherwise, add DISK_SNAPSHOT_PERIOD + next_snapshot_height = current_snapshot_height + DISK_SNAPSHOT_PERIOD; + } + + if (next_snapshot_height > stop_index->nHeight) { + break; + } + + const CBlockIndex* next_snapshot_index = stop_index->GetAncestor(next_snapshot_height); + if (!next_snapshot_index) { + break; + } + + snapshot_blocks.push_back(next_snapshot_index); + current_snapshot_height = next_snapshot_height; + } + + return snapshot_blocks; +} + +bool CDeterministicMNManager::VerifySnapshotPair( + const CBlockIndex* from_index, const CBlockIndex* to_index, const CDeterministicMNList& from_snapshot, + const CDeterministicMNList& to_snapshot, RecalcDiffsResult& result) +{ + AssertLockHeld(::cs_main); + + // Verify this snapshot pair by applying all stored diffs sequentially + CDeterministicMNList test_list = from_snapshot; + + try { + for (int nHeight = from_index->nHeight + 1; nHeight <= to_index->nHeight; ++nHeight) { + const CBlockIndex* pIndex = to_index->GetAncestor(nHeight); + if (!pIndex) { + result.verification_errors.push_back(strprintf("Failed to get ancestor at height %d", nHeight)); + return false; + } + + CDeterministicMNListDiff diff; + if (!m_evoDb.Read(std::make_pair(DB_LIST_DIFF, pIndex->GetBlockHash()), diff)) { + result.verification_errors.push_back(strprintf("Failed to read diff at height %d", nHeight)); + return false; + } + + diff.nHeight = nHeight; + test_list.ApplyDiff(pIndex, diff); + } + } catch (const std::exception& e) { + result.verification_errors.push_back(strprintf("Exception during verification: %s", e.what())); + return false; + } + + // Verify that applying all diffs results in the target snapshot + bool is_snapshot_pair_valid = test_list.IsEqual(to_snapshot); + + if (is_snapshot_pair_valid) { + result.snapshots_verified++; + } else { + result.verification_errors.push_back( + strprintf("Verification failed between snapshots at heights %d and %d: " + "Applied diffs do not match target snapshot", + from_index->nHeight, to_index->nHeight)); + } + + return is_snapshot_pair_valid; +} + +std::vector> CDeterministicMNManager::RepairSnapshotPair( + const CBlockIndex* from_index, const CBlockIndex* to_index, const CDeterministicMNList& from_snapshot, + const CDeterministicMNList& to_snapshot, BuildListFromBlockFunc build_list_func, RecalcDiffsResult& result) +{ + AssertLockHeld(::cs_main); + + CDeterministicMNList current_list = from_snapshot; + // Temporary storage for recalculated diffs (one per block in this snapshot interval) + std::vector> temp_diffs; + temp_diffs.reserve(to_index->nHeight - from_index->nHeight); + + LogPrintf("CDeterministicMNManager::%s -- Repairing: recalculating diffs between snapshots at heights %d and %d\n", + __func__, from_index->nHeight, to_index->nHeight); + + try { + for (int nHeight = from_index->nHeight + 1; nHeight <= to_index->nHeight; ++nHeight) { + const CBlockIndex* pIndex = to_index->GetAncestor(nHeight); + + // Read the actual block from disk + CBlock block; + if (!node::ReadBlockFromDisk(block, pIndex, Params().GetConsensus())) { + result.repair_errors.push_back(strprintf("CRITICAL: Failed to read block at height %d. " + "Cannot repair - full reindex required.", nHeight)); + return {}; // Critical error - cannot continue repair + } + + // Use a dummy coins view to avoid UTXO lookups. At chain tip, coins from + // historical blocks may already be spent. Since these blocks were fully + // validated when originally connected, we don't need to re-verify coin + // availability - we only need to extract special transactions. + CCoinsView view_dummy; + CCoinsViewCache view(&view_dummy); + + // Build the new list by processing this block's special transactions + // Starting from current_list (our trusted state), not from corrupted diffs + CDeterministicMNList next_list; + BlockValidationState state; + if (!build_list_func(block, pIndex->pprev, current_list, view, false, state, next_list)) { + result.repair_errors.push_back( + strprintf("CRITICAL: Failed to build list for block at height %d: %s. " + "Cannot repair - full reindex required.", nHeight, state.ToString())); + return {}; // Critical error - cannot continue repair + } + + // Set the correct block hash + next_list.SetBlockHash(pIndex->GetBlockHash()); + + // Calculate the diff between current and next + CDeterministicMNListDiff recalc_diff = current_list.BuildDiff(next_list); + recalc_diff.nHeight = nHeight; + // Store in temporary vector for this snapshot pair + temp_diffs.emplace_back(pIndex->GetBlockHash(), recalc_diff); + + // Move forward + current_list = std::move(next_list); + } + + // Verify that applying all diffs results in the target snapshot + if (current_list.IsEqual(to_snapshot)) { + LogPrintf("CDeterministicMNManager::%s -- Successfully recalculated %d diffs between heights %d and %d\n", + __func__, temp_diffs.size(), from_index->nHeight, to_index->nHeight); + return temp_diffs; // Success - return recalculated diffs + } else { + result.repair_errors.push_back( + strprintf("CRITICAL: Recalculation failed between snapshots at heights %d and %d: " + "Applied diffs do not match target snapshot. Cannot repair - full reindex required.", + from_index->nHeight, to_index->nHeight)); + return {}; // Failed verification - return empty vector + } + } catch (const std::exception& e) { + result.repair_errors.push_back(strprintf("CRITICAL: Exception during recalculation: %s. " + "Cannot repair - full reindex required.", e.what())); + return {}; // Exception - return empty vector + } +} + +void CDeterministicMNManager::WriteRepairedDiffs( + const std::vector>& recalculated_diffs, RecalcDiffsResult& result) +{ + AssertLockNotHeld(cs); + + if (recalculated_diffs.empty()) { + return; + } + + CDBBatch batch(m_evoDb.GetRawDB()); + const size_t BATCH_SIZE_THRESHOLD = 1 << 24; // 16MB + size_t diffs_written = 0; + + LogPrintf("CDeterministicMNManager::%s -- Writing %d repaired diffs to database...\n", + __func__, recalculated_diffs.size()); + + for (const auto& [block_hash, diff] : recalculated_diffs) { + batch.Write(std::make_pair(DB_LIST_DIFF, block_hash), diff); + diffs_written++; + + // Write batch when it gets too large + if (batch.SizeEstimate() >= BATCH_SIZE_THRESHOLD) { + LogPrintf("CDeterministicMNManager::%s -- Flushing batch (%d diffs written so far)...\n", + __func__, diffs_written); + m_evoDb.GetRawDB().WriteBatch(batch); + batch.Clear(); + } + } + + // Write any remaining diffs in the batch + if (batch.SizeEstimate() > 0) { + LogPrintf("CDeterministicMNManager::%s -- Writing final batch...\n", __func__); + m_evoDb.GetRawDB().WriteBatch(batch); + batch.Clear(); + } + + // Clear caches for repaired diffs so next read gets fresh data from disk + // Must clear both diff cache and list cache since lists were built from old diffs + LOCK(cs); + for (const auto& [block_hash, diff] : recalculated_diffs) { + mnListDiffsCache.erase(block_hash); + mnListsCache.erase(block_hash); + } + + LogPrintf("CDeterministicMNManager::%s -- Successfully repaired %d diffs (caches cleared)\n", __func__, + recalculated_diffs.size()); +} diff --git a/src/evo/deterministicmns.h b/src/evo/deterministicmns.h index fe5a4f1dd7876..b01fbd5f8f7ed 100644 --- a/src/evo/deterministicmns.h +++ b/src/evo/deterministicmns.h @@ -34,6 +34,8 @@ class CEvoDB; class CSimplifiedMNList; class CSimplifiedMNListEntry; class CMasternodeMetaMan; +class ChainstateManager; +class CSpecialTxProcessor; class TxValidationState; struct RPCResult; @@ -436,6 +438,46 @@ class CDeterministicMNList return GetMN(p->first); } + // Compare two masternode lists for equality, ignoring non-deterministic members. + // Non-deterministic members (nTotalRegisteredCount, internalId) can differ between + // nodes due to different sync histories, but don't affect consensus validity. + bool IsEqual(const CDeterministicMNList& rhs) const + { + // Compare deterministic metadata + if (blockHash != rhs.blockHash || + nHeight != rhs.nHeight || + mnUniquePropertyMap != rhs.mnUniquePropertyMap) { + return false; + } + + // Compare map sizes (actual entries compared below) + // Note: Not comparing nTotalRegisteredCount (non-deterministic) + if (mnMap.size() != rhs.mnMap.size() || + mnInternalIdMap.size() != rhs.mnInternalIdMap.size()) { + return false; + } + + // Compare each masternode entry + for (const auto& [proTxHash, dmn] : mnMap) { + auto dmn_rhs = rhs.mnMap.find(proTxHash); + if (dmn_rhs == nullptr) { + return false; + } + + // Compare deterministic masternode fields + // Note: Not comparing internalId (non-deterministic) + if (dmn->proTxHash != dmn_rhs->get()->proTxHash || + dmn->collateralOutpoint != dmn_rhs->get()->collateralOutpoint || + dmn->nOperatorReward != dmn_rhs->get()->nOperatorReward || + dmn->nType != dmn_rhs->get()->nType || + // Use SerializeHash for pdmnState to avoid enumerating all state fields + SerializeHash(*dmn->pdmnState) != SerializeHash(*dmn_rhs->get()->pdmnState)) { + return false; + } + } + return true; + } + private: template [[nodiscard]] uint256 GetUniquePropertyHash(const T& v) const @@ -678,6 +720,31 @@ class CDeterministicMNManager void DoMaintenance() EXCLUSIVE_LOCKS_REQUIRED(!cs, !cs_cleanup); + // Recalculate and optionally repair diffs between snapshots + struct RecalcDiffsResult { + int start_height{0}; + int stop_height{0}; + int diffs_recalculated{0}; + int snapshots_verified{0}; + std::vector verification_errors; + std::vector repair_errors; + }; + + // Callback type for building a new MN list from a block + using BuildListFromBlockFunc = std::function pindexPrev, + const CDeterministicMNList& prevList, + const CCoinsViewCache& view, + bool debugLogs, + BlockValidationState& state, + CDeterministicMNList& mnListRet)>; + + [[nodiscard]] RecalcDiffsResult RecalculateAndRepairDiffs( + const CBlockIndex* start_index, const CBlockIndex* stop_index, + ChainstateManager& chainman, BuildListFromBlockFunc build_list_func, + bool repair) EXCLUSIVE_LOCKS_REQUIRED(!cs, ::cs_main); + // Migration support for nVersion-first CDeterministicMNStateDiff format [[nodiscard]] bool IsMigrationRequired() const EXCLUSIVE_LOCKS_REQUIRED(!cs, ::cs_main); [[nodiscard]] bool MigrateLegacyDiffs(const CBlockIndex* const tip_index) EXCLUSIVE_LOCKS_REQUIRED(!cs, ::cs_main); @@ -685,6 +752,19 @@ class CDeterministicMNManager private: void CleanupCache(int nHeight) EXCLUSIVE_LOCKS_REQUIRED(cs); CDeterministicMNList GetListForBlockInternal(gsl::not_null pindex) EXCLUSIVE_LOCKS_REQUIRED(cs); + + // Helper methods for RecalculateAndRepairDiffs + std::vector CollectSnapshotBlocks(const CBlockIndex* start_index, const CBlockIndex* stop_index, + const Consensus::Params& consensus_params) EXCLUSIVE_LOCKS_REQUIRED(::cs_main); + bool VerifySnapshotPair(const CBlockIndex* from_index, const CBlockIndex* to_index, const CDeterministicMNList& from_snapshot, + const CDeterministicMNList& to_snapshot, RecalcDiffsResult& result) + EXCLUSIVE_LOCKS_REQUIRED(::cs_main); + std::vector> RepairSnapshotPair( + const CBlockIndex* from_index, const CBlockIndex* to_index, const CDeterministicMNList& from_snapshot, + const CDeterministicMNList& to_snapshot, BuildListFromBlockFunc build_list_func, RecalcDiffsResult& result) + EXCLUSIVE_LOCKS_REQUIRED(::cs_main); + void WriteRepairedDiffs(const std::vector>& recalculated_diffs, + RecalcDiffsResult& result) EXCLUSIVE_LOCKS_REQUIRED(!cs); }; bool CheckProRegTx(CDeterministicMNManager& dmnman, const CTransaction& tx, gsl::not_null pindexPrev, TxValidationState& state, const CCoinsViewCache& view, bool check_sigs); diff --git a/src/evo/specialtxman.cpp b/src/evo/specialtxman.cpp index 35d3fe14bc431..a788bc22437fb 100644 --- a/src/evo/specialtxman.cpp +++ b/src/evo/specialtxman.cpp @@ -176,10 +176,24 @@ bool CSpecialTxProcessor::BuildNewListFromBlock(const CBlock& block, gsl::not_nu BlockValidationState& state, CDeterministicMNList& mnListRet) { AssertLockHeld(cs_main); + CDeterministicMNList oldList = m_dmnman.GetListForBlock(pindexPrev); + return RebuildListFromBlock(block, pindexPrev, oldList, view, debugLogs, state, mnListRet); +} + +bool CSpecialTxProcessor::RebuildListFromBlock(const CBlock& block, gsl::not_null pindexPrev, + const CDeterministicMNList& prevList, const CCoinsViewCache& view, + bool debugLogs, BlockValidationState& state, + CDeterministicMNList& mnListRet) +{ + AssertLockHeld(cs_main); + + // Verify that prevList either represents an empty/initial state (default-constructed), + // or it matches the previous block's hash. + assert(prevList == CDeterministicMNList() || prevList.GetBlockHash() == pindexPrev->GetBlockHash()); int nHeight = pindexPrev->nHeight + 1; - CDeterministicMNList oldList = m_dmnman.GetListForBlock(pindexPrev); + CDeterministicMNList oldList = prevList; CDeterministicMNList newList = oldList; newList.SetBlockHash(uint256()); // we can't know the final block hash, so better not return a (invalid) block hash newList.SetHeight(nHeight); @@ -235,13 +249,18 @@ bool CSpecialTxProcessor::BuildNewListFromBlock(const CBlock& block, gsl::not_nu dmn->collateralOutpoint = proTx.collateralOutpoint; } - Coin coin; - CAmount expectedCollateral = GetMnType(proTx.nType).collat_amount; - if (!proTx.collateralOutpoint.hash.IsNull() && (!view.GetCoin(dmn->collateralOutpoint, coin) || - coin.IsSpent() || coin.out.nValue != expectedCollateral)) { - // should actually never get to this point as CheckProRegTx should have handled this case. - // We do this additional check nevertheless to be 100% sure - return state.Invalid(BlockValidationResult::BLOCK_CONSENSUS, "bad-protx-collateral"); + // Complain about spent collaterals only when we process the tip. + // This is safe because blocks below the tip were verified + // when they were connected initially. + if (!view.GetBestBlock().IsNull()) { + Coin coin; + CAmount expectedCollateral = GetMnType(proTx.nType).collat_amount; + if (!proTx.collateralOutpoint.hash.IsNull() && (!view.GetCoin(dmn->collateralOutpoint, coin) || + coin.IsSpent() || coin.out.nValue != expectedCollateral)) { + // should actually never get to this point as CheckProRegTx should have handled this case. + // We do this additional check nevertheless to be 100% sure + return state.Invalid(BlockValidationResult::BLOCK_CONSENSUS, "bad-protx-collateral"); + } } auto replacedDmn = newList.GetMNByCollateral(dmn->collateralOutpoint); diff --git a/src/evo/specialtxman.h b/src/evo/specialtxman.h index 84f74cd06e855..461fa8d222c18 100644 --- a/src/evo/specialtxman.h +++ b/src/evo/specialtxman.h @@ -79,6 +79,13 @@ class CSpecialTxProcessor const CCoinsViewCache& view, bool debugLogs, BlockValidationState& state, CDeterministicMNList& mnListRet) EXCLUSIVE_LOCKS_REQUIRED(cs_main); + // Variant that takes an explicit starting list instead of loading from GetListForBlock + // Used for rebuilding diffs from trusted snapshots + bool RebuildListFromBlock(const CBlock& block, gsl::not_null pindexPrev, + const CDeterministicMNList& prevList, const CCoinsViewCache& view, bool debugLogs, + BlockValidationState& state, CDeterministicMNList& mnListRet) + EXCLUSIVE_LOCKS_REQUIRED(cs_main); + private: bool CheckCreditPoolDiffForBlock(const CBlock& block, const CBlockIndex* pindex, const CCbTx& cbTx, BlockValidationState& state) EXCLUSIVE_LOCKS_REQUIRED(::cs_main); diff --git a/src/rpc/evo.cpp b/src/rpc/evo.cpp index 1ddd1cfd9980a..12bd74cc79ddc 100644 --- a/src/rpc/evo.cpp +++ b/src/rpc/evo.cpp @@ -1752,6 +1752,175 @@ static RPCHelpMan protx_listdiff() }; } +// Helper function for evodb verify/repair commands +static UniValue evodb_verify_or_repair_impl(const JSONRPCRequest& request, bool repair) +{ + const NodeContext& node = EnsureAnyNodeContext(request.context); + ChainstateManager& chainman = EnsureChainman(node); + CDeterministicMNManager& dmnman = *CHECK_NONFATAL(node.dmnman); + CChainstateHelper& chain_helper = *CHECK_NONFATAL(node.chain_helper); + + LOCK(::cs_main); + + const CBlockIndex* start_index; + const CBlockIndex* stop_index; + + // Default to DIP0003 activation height if startBlock not specified + if (request.params[0].isNull()) { + const auto& consensus_params = Params().GetConsensus(); + start_index = chainman.ActiveChain()[consensus_params.DIP0003Height]; + if (!start_index) { + throw JSONRPCError(RPC_INTERNAL_ERROR, "Cannot find DIP0003 activation block"); + } + } else { + uint256 start_block_hash = ParseBlock(request.params[0], chainman, "startBlock"); + start_index = chainman.m_blockman.LookupBlockIndex(start_block_hash); + if (!start_index) { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Start block not found"); + } + } + + // Default to chain tip if stopBlock not specified + if (request.params[1].isNull()) { + stop_index = chainman.ActiveChain().Tip(); + if (!stop_index) { + throw JSONRPCError(RPC_INTERNAL_ERROR, "Cannot find chain tip"); + } + } else { + uint256 stop_block_hash = ParseBlock(request.params[1], chainman, "stopBlock"); + stop_index = chainman.m_blockman.LookupBlockIndex(stop_block_hash); + if (!stop_index) { + throw JSONRPCError(RPC_INVALID_PARAMETER, "Stop block not found"); + } + } + + int start_height = start_index->nHeight; + int stop_height = stop_index->nHeight; + + // Validation + if (stop_height < start_height) { + throw JSONRPCError(RPC_INVALID_PARAMETER, "stopBlock must be >= startBlock"); + } + + // Create a callback that wraps CSpecialTxProcessor::BuildNewListFromBlock + // NO_THREAD_SAFETY_ANALYSIS: cs_main is held by the calling function (evodb_verify_or_repair_impl) + auto build_list_func = [&chain_helper](const CBlock& block, gsl::not_null pindexPrev, + const CDeterministicMNList& prevList, const CCoinsViewCache& view, + bool debugLogs, BlockValidationState& state, + CDeterministicMNList& mnListRet) NO_THREAD_SAFETY_ANALYSIS -> bool { + return chain_helper.special_tx->RebuildListFromBlock(block, pindexPrev, prevList, view, debugLogs, state, mnListRet); + }; + + // Call the dmnman method to do the work + auto recalc_result = dmnman.RecalculateAndRepairDiffs(start_index, stop_index, chainman, build_list_func, repair); + + // Convert result to UniValue + UniValue result(UniValue::VOBJ); + UniValue verification_errors(UniValue::VARR); + + for (const auto& error : recalc_result.verification_errors) { + verification_errors.push_back(error); + } + + result.pushKV("startHeight", recalc_result.start_height); + result.pushKV("stopHeight", recalc_result.stop_height); + result.pushKV("diffsRecalculated", recalc_result.diffs_recalculated); + result.pushKV("snapshotsVerified", recalc_result.snapshots_verified); + result.pushKV("verificationErrors", verification_errors); + + // Only include repair errors if we're in repair mode + if (repair) { + UniValue repair_errors(UniValue::VARR); + for (const auto& error : recalc_result.repair_errors) { + repair_errors.push_back(error); + } + result.pushKV("repairErrors", repair_errors); + } + + return result; +} + +static RPCHelpMan evodb_verify() +{ + return RPCHelpMan{"evodb verify", + "\nVerifies evodb diff records between specified block heights.\n" + "Checks that all diffs applied between snapshots in the range match the saved snapshots in evodb.\n" + "This is a read-only operation that does not modify the database.\n" + "If no heights are specified, defaults to the full range from DIP0003 activation to chain tip.\n", + { + {"startBlock", RPCArg::Type::NUM, RPCArg::Optional::OMITTED, "The starting block height (defaults to DIP0003 activation height)."}, + {"stopBlock", RPCArg::Type::NUM, RPCArg::Optional::OMITTED, "The ending block height (defaults to current chain tip)."}, + }, + RPCResult{ + RPCResult::Type::OBJ, "", "", + { + {RPCResult::Type::NUM, "startHeight", "Actual starting block height (may differ from input if clamped to DIP0003 activation)"}, + {RPCResult::Type::NUM, "stopHeight", "Ending block height"}, + {RPCResult::Type::NUM, "diffsRecalculated", "Number of diffs recalculated (always 0 for verify-only mode)"}, + {RPCResult::Type::NUM, "snapshotsVerified", "Number of snapshot pairs that passed verification"}, + {RPCResult::Type::ARR, "verificationErrors", "List of verification errors (empty if verification passed)", + { + {RPCResult::Type::STR, "", "Error message"}, + } + }, + } + }, + RPCExamples{ + HelpExampleCli("evodb verify", "") + + HelpExampleCli("evodb verify", "1000 2000") + + HelpExampleRpc("evodb", "\"verify\"") + + HelpExampleRpc("evodb", "\"verify\", 1000, 2000") + }, + [&](const RPCHelpMan& self, const JSONRPCRequest& request) -> UniValue +{ + return evodb_verify_or_repair_impl(request, false); +}, + }; +} + +static RPCHelpMan evodb_repair() +{ + return RPCHelpMan{"evodb repair", + "\nRepairs corrupted evodb diff records between specified block heights.\n" + "First verifies all diffs applied between snapshots in the range.\n" + "If verification fails, recalculates diffs from blockchain data and replaces corrupted records.\n" + "If no heights are specified, defaults to the full range from DIP0003 activation to chain tip.\n", + { + {"startBlock", RPCArg::Type::NUM, RPCArg::Optional::OMITTED, "The starting block height (defaults to DIP0003 activation height)."}, + {"stopBlock", RPCArg::Type::NUM, RPCArg::Optional::OMITTED, "The ending block height (defaults to current chain tip)."}, + }, + RPCResult{ + RPCResult::Type::OBJ, "", "", + { + {RPCResult::Type::NUM, "startHeight", "Actual starting block height (may differ from input if clamped to DIP0003 activation)"}, + {RPCResult::Type::NUM, "stopHeight", "Ending block height"}, + {RPCResult::Type::NUM, "diffsRecalculated", "Number of diffs successfully recalculated and written to database"}, + {RPCResult::Type::NUM, "snapshotsVerified", "Number of snapshot pairs that passed verification"}, + {RPCResult::Type::ARR, "verificationErrors", "Errors encountered during verification phase (empty if verification passed)", + { + {RPCResult::Type::STR, "", "Error message"}, + } + }, + {RPCResult::Type::ARR, "repairErrors", "Critical errors encountered during repair phase (non-empty means full reindex required)", + { + {RPCResult::Type::STR, "", "Error message"}, + } + }, + } + }, + RPCExamples{ + HelpExampleCli("evodb repair", "") + + HelpExampleCli("evodb repair", "1000 2000") + + HelpExampleRpc("evodb", "\"repair\"") + + HelpExampleRpc("evodb", "\"repair\", 1000, 2000") + }, + [&](const RPCHelpMan& self, const JSONRPCRequest& request) -> UniValue +{ + return evodb_verify_or_repair_impl(request, true); +}, + }; +} + static RPCHelpMan protx_help() { return RPCHelpMan{ @@ -1919,6 +2088,8 @@ void RegisterEvoRPCCommands(CRPCTable& tableRPC) {"evo", &protx_help}, {"evo", &protx_diff}, {"evo", &protx_listdiff}, + {"hidden", &evodb_verify}, + {"hidden", &evodb_repair}, }; static const CRPCCommand commands_wallet[]{ {"evo", &protx_list},