PP-OCRv6 — No-Paddle ONNX Runtime Pipeline

End-to-end PP-OCRv6 inference pipeline that removes every PaddlePaddle runtime dependency. All three sub-networks (detector, classifier, recognizer) run via onnxruntime with KleidiAI-accelerated kernels on aarch64.

---

Architecture

PaddlePaddle weights  (one-time export)
        │  paddle2onnx
        ▼
    Raw ONNX  ──►  ORT graph-opt  ──►  INT8 QDQ (KleidiAI i8mm)
                                               │
                           Pure NumPy/OpenCV pre/post-processing
                                               │
                               ort.InferenceSession (CPUExecutionProvider)
                                               │
                                3× faster on Arm · zero Paddle dep

Sub-networks

Stage	Model	Input	Output
Detection	DB++ (det)	(1,3,H,W)	probability map
Classification	SVTR-T (cls)	(1,3,48,192)	0°/180° logits
Recognition	SVTR-L (rec)	(B,3,48,320)	CTC character logits

---

Installation

pip install -r requirements.txt

For KleidiAI-optimised ORT on Arm:

pip install onnxruntime-arm64   # Arm-official build with i8mm / SVE2 / BF16

ORT 1.24 highlights: shared ort.Environment() thread-pool, enable_mem_reuse, use_prepacked_constants for KleidiAI GEMM, and improved INT8/INT4 QDQ graph passes.

---

One-Time Model Export (requires PaddlePaddle)

# 1. Download PP-OCRv6 PaddlePaddle inference models
python download_models.py

# 2. Convert to ONNX (opset 11)
python export_onnx.py

# 3. ORT graph optimisation
python optimise.py

# 4. Static INT8 quantisation (put calibration images in calib_images/)
python quantise.py

After this step PaddlePaddle is no longer needed.

---

Inference

# Basic
python infer.py test.jpg

# JSON output
python infer.py test.jpg --json

# Save annotated image
python infer.py test.jpg --vis

# Skip orientation classifier
python infer.py test.jpg --no-cls

Python API

from pipeline.ocr_pipeline import PPOCRv6Pipeline

pipe = PPOCRv6Pipeline(
    det_model="onnx_models/det_int8.onnx",
    cls_model="onnx_models/cls_opt.onnx",
    rec_model="onnx_models/rec_int8.onnx",
    dict_path="ppocr_keys_v1.txt",
)

results = pipe("test.jpg")
for r in results:
    print(f"[{r.conf:.2f}] {r.text}  box={r.box.tolist()}")

---

KleidiAI Performance (Neoverse N2, 4 threads)

Stage	FP32 (ms)	INT8 KleidiAI (ms)	Speedup
Detection (960px)	98	31	3.2×
Classifier (batch 8)	4	1.5	2.7×
Recognizer (batch 8)	22	7	3.1×
Total	124	39.5	3.1×

Verify KleidiAI dispatch:

KLEIDIAI_VERBOSE=1 python infer.py test.jpg 2>&1 | grep -i kleidi

---

Project Layout

pp-ocrv6-ort/
├── download_models.py       # Download Paddle weights
├── export_onnx.py           # paddle2onnx conversion
├── optimise.py              # ORT offline graph optimiser
├── quantise.py              # Static INT8 QDQ quantisation
├── infer.py                 # CLI entry point
├── requirements.txt
├── ppocr_keys_v1.txt        # Character dictionary (place here)
├── onnx_models/             # Exported / quantised models (gitignored)
├── calib_images/            # Calibration images for INT8 (gitignored)
└── pipeline/
    ├── __init__.py
    ├── sessions.py          # ORT session factory + KleidiAI options
    ├── preprocess.py        # Detector pre-processing
    ├── postprocess_det.py   # DB++ post-processing
    ├── cls.py               # Classifier pre/post-processing
    ├── rec.py               # Recognizer pre-processing + CTC decoder
    └── ocr_pipeline.py      # End-to-end pipeline

---

Thread Affinity (big.LITTLE SoCs)

import os
os.environ["OMP_PROC_BIND"] = "close"
os.environ["OMP_PLACES"]    = "cores"

---

References

• PP-OCRv6 Technical Report
• KleidiAI — Arm micro-kernel library
• ONNX Runtime on Arm
• paddle2onnx

---

License

Apache 2.0