Skip to content

richard202605/midnight-proof-server-debugging

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
 
 
 
 

Repository files navigation

When Proofs Fail: Debugging Proof Server Errors & ZK Generation Failures on Midnight

A comprehensive debugging guide and diagnostic toolkit for Midnight Network proof server issues. Covers the 5 most common failure modes with specific diagnosis steps, code fixes, and an automated diagnostic script.

What's Included

📖 Tutorial Content (README below)

  • How Midnight's proof server architecture works
  • 5 failure modes with symptoms, diagnosis, and fixes
  • Diagnostic workflow (6-step checklist)
  • Prevention checklist
  • Quick reference table

🛠️ Diagnostic Tools

  • scripts/diagnose_proof_server.sh — Automated bash diagnostic script (container, health, port, memory, disk, network)
  • scripts/version-check.ts — TypeScript version alignment checker (proof server vs SDK vs Compact compiler)

📋 Code Examples

  • examples/prewarm-proof-server.ts — Proof server warm-up pattern
  • examples/health-check-docker-compose.yml — Docker Compose with health checks
  • examples/sdk-timeout-config.ts — Proper timeout configuration

Quick Start

# Run the diagnostic script
chmod +x scripts/diagnose_proof_server.sh
./scripts/diagnose_proof_server.sh localhost 6300 --verbose

# Check version alignment
npx ts-node scripts/version-check.ts

How Midnight's Proof Server Works

Before debugging, you need to understand the architecture. Midnight's proof server is a separate service (typically running in Docker) that:

  1. Receives proof generation requests from your application
  2. Downloads and caches ZK parameters on first use (~30MB)
  3. Generates zero-knowledge proofs using your circuit's witness data
  4. Returns the proof to your application for transaction submission

The proof server exposes two main endpoints: /check (health check) and /prove (proof generation). Your SDK communicates with it via HTTP.

Your App → SDK → HTTP → Proof Server (Docker) → ZK Parameters → Proof → Transaction Submission → Node

A failure at any point in this chain can manifest as a timeout, rejection, or silent invalid proof.


Failure Mode 1: Proof Server Not Responding

Symptoms

  • Application hangs on proof generation
  • ECONNREFUSED or ETIMEDOUT errors in your logs
  • SDK retry mechanism keeps retrying with no success

Diagnosis

Start by checking if the Docker container is running:

# Check container status
docker ps | grep proof-server

# If not running, check why it stopped
docker logs midnight-proof-server --tail 50

Common log messages and what they mean:

# Port conflict — another service is using the same port
Error: bind EADDRINUSE 0.0.0.0:6300

# Missing ZK parameters — first run without pre-downloaded params
Error: ENOENT: no such file or directory, open '/params/circuit_0.params'

# Memory limit — container OOM killed
Killed

The Fix

Port conflict: Change the host port mapping in your Docker run command:

docker run -d --name midnight-proof-server \
  -p 6301:6300 \
  midnightntwrk/proof-server:latest

Missing parameters: Pre-download the ZK parameters or mount a persistent volume:

docker run -d --name midnight-proof-server \
  -v ./proof-params:/params \
  -p 6300:6300 \
  midnightntwrk/proof-server:latest

Memory limits: Increase the container memory allocation:

docker run -d --name midnight-proof-server \
  --memory=2g \
  -p 6300:6300 \
  midnightntwrk/proof-server:latest

Verify It's Working

curl http://localhost:6300/check
# Should return: {"status":"ok"} or similar

Failure Mode 2: Proof Timeout (First Proof Takes Forever)

Symptoms

  • First proof generation takes 2-5 minutes
  • Subsequent proofs are fast (under 10 seconds)
  • SDK may throw a timeout error before the proof completes

What's Actually Happening

The first proof triggers a download of ZK parameters (~30MB). These parameters are specific to your circuit and are cached after the first download. On a slow connection or with large circuits, this can take several minutes.

The Fix

Increase SDK timeout for first proof:

const proofConfig = {
  timeout: 300_000, // 5 minutes for first proof
  retries: 1,       // Don't retry — if it fails, diagnose
};

Pre-warm the proof server: Generate a dummy proof during application startup:

async function warmUpProofServer() {
  console.log('Warming up proof server (first proof may take 2-5 min)...');
  try {
    await contract.circuits.dummyCircuit(ctx);
    console.log('Proof server warmed up.');
  } catch (e) {
    console.log('Warm-up complete (error expected):', e.message);
  }
}

Failure Mode 3: Proof Rejection from Wire Format Mismatch

Symptoms

  • Proof generates successfully (no error in proof generation)
  • Transaction submission fails with Error 1010 or similar
  • Proof bytes look valid in logs but node rejects them

What's Actually Happening

The proof server generates a proof in a specific wire format. If the format expected by the node doesn't match what the proof server produces, the node rejects it. This is almost always a version mismatch.

The Fix

Pin all versions to the same release:

{
  "dependencies": {
    "@midnight-ntwrk/sdk": "0.29.0",
    "@midnight-ntwrk/compact": "0.29.0"
  }
}
docker pull midnightntwrk/proof-server:0.29.0
docker run -d --name midnight-proof-server \
  -p 6300:6300 \
  midnightntwrk/proof-server:0.29.0

Failure Mode 4: Version Mismatch Between Proof Server and Ledger

Symptoms

  • Proofs generate but fail validation
  • Error messages reference unexpected proof format versions
  • Works on one network (testnet) but fails on another (devnet)

The Fix

Use the proof server version that matches your target network:

# For testnet
docker pull midnightntwrk/proof-server:testnet-0.29.0

# For devnet
docker pull midnightntwrk/proof-server:devnet-0.29.0

Check the Midnight Network status page for current version requirements.


Failure Mode 5: Proof Server Health Check Fails

Symptoms

  • /check endpoint returns non-200 status
  • SDK throws "proof server unavailable" before attempting proof generation
  • Intermittent failures — sometimes works, sometimes doesn't

Common Health Check Responses

Response Meaning Action
{"status":"ok"} Healthy No action needed
{"status":"degraded","message":"low memory"} Under pressure Increase container memory
{"status":"error","message":"params not found"} Missing parameters Re-download or remount volume
Connection refused Server not running Check Docker logs
503 Service Unavailable Server overloaded Reduce concurrent proof requests

The Fix

Add a health check to your Docker Compose:

services:
  proof-server:
    image: midnightntwrk/proof-server:0.29.0
    ports:
      - "6300:6300"
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:6300/check"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 60s

Diagnostic Workflow

When your proof server misbehaves, run through this sequence:

  1. Check Container Statusdocker ps | grep proof-server
  2. Verify Healthcurl http://localhost:6300/check
  3. Check Version Alignmentdocker exec midnight-proof-server cat /version.txt
  4. Test with Minimal Circuit — Use the simplest possible circuit
  5. Check Resourcesdocker stats midnight-proof-server --no-stream
  6. Check Network Connectivitydocker exec midnight-proof-server curl -v https://testnet.midnight.network/status

Or run the automated script: ./scripts/diagnose_proof_server.sh


Prevention Checklist

  1. Pin Docker image tags. Never use :latest in production.
  2. Mount persistent volumes for ZK parameters. The 30MB download only happens once if you persist the parameters.
  3. Set appropriate timeouts. First proof: 5 minutes. Subsequent proofs: 30 seconds.
  4. Monitor proof server health. Add Docker health checks and alert on non-200 responses.
  5. Keep versions synchronized. Proof server, SDK, Compact compiler, and target network must all be on compatible versions.
  6. Pre-warm in development. Generate a dummy proof after starting the proof server to trigger parameter download.

Quick Reference

Symptom Likely Cause First Action
Connection refused Container not running docker ps, check logs
First proof slow (2-5 min) ZK parameter download Wait or pre-download
Timeout on all proofs Resource exhaustion Check memory/disk
Proof generates but rejected Version mismatch Align versions
Intermittent failures Resource pressure or network Check health, add retries
Health check returns error Misconfiguration Check logs, verify params

Resources


This guide is based on real proof server debugging sessions. If you've hit a proof server failure mode not listed here, share it on the Midnight Discord.

About

Debugging Midnight proof server errors: 5 failure modes, diagnostic tools, version alignment checker

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors