IMMACULATE

This repository contains the anonymous implementation of IMMACULATE: A Practical LLM Auditing Framework via Verifiable Computation.

The release keeps the three experiment codebases intact:

Directory	Purpose	Reference precision
inference/	Modified vLLM inference and trace collection	Model-dependent
transformer-4.57.3/	Verification for LLaMA 3 and Qwen 3 models	FP16
transformers-4.39.3/	Verification for DeepSeek-V2-Lite	FP32

The two Transformers versions must be installed in separate environments. All commands below assume that IMMACULATE_CACHE_DIR points to a writable directory. Models and datasets are downloaded or read from that directory, while traces are stored below $IMMACULATE_CACHE_DIR/logs.

export IMMACULATE_CACHE_DIR=/path/to/immaculate-cache

GPU selection is left to the caller:

export CUDA_VISIBLE_DEVICES=0,1

1. Inference

Create an environment with a CUDA-enabled PyTorch installation, then install the modified vLLM tree and the dataset dependency:

cd inference
python3 -m pip install datasets
bash build.sh

Dense models use tensor parallelism of degree 2:

python3 main_dense.py \
  --model meta-llama/Meta-Llama-3-70B-Instruct \
  --log_dir llama70b

MoE models use tensor parallelism of degree 1:

python3 main_moe.py \
  --model Qwen/Qwen3-30B-A3B-Instruct-2507 \
  --log_dir qwen3moe

Pass --quantization fp8 and use a distinct --log_dir for the FP8 deployment. precision.sh lists the model/deployment combinations used by the experiments.

The paper evaluation uses the first 600 prompts from each of GSM8K, MBPP, and WebQuestions.

Each prompt is stored below:

$IMMACULATE_CACHE_DIR/logs/<log_dir>/<first-16-hex-of-prompt-md5>/

Dense traces contain:

topk_indices.pkl
topk_logits.pkl
output_token_ids.npy
digest.sha256

MoE traces additionally contain expert_ids.pkl and input_token_ids.npy. The MD5 prefix is only a directory lookup key.

digest.sha256 is one SHA-256 digest over the trace files in the fixed order defined by the corresponding inference entry point. The digest input uses a domain separator and, for every file, an eight-byte big-endian filename length, the UTF-8 filename, an eight-byte big-endian file length, and the raw file bytes. This digest protects stored artifact integrity; it is not a standalone implementation of the paper's online commitment or verifiable computation protocol.

2. LLaMA and Qwen verification

Use a separate environment and install this Transformers fork:

cd transformer-4.57.3
python3 -m pip install -e .
python3 -m pip install accelerate datasets

The reference execution uses FP16:

python3 main_dense.py \
  --model meta-llama/Meta-Llama-3-70B-Instruct \
  --fp16 \
  --log_dir llama70b

python3 main_moe.py \
  --model Qwen/Qwen3-30B-A3B-Instruct-2507 \
  --fp16 \
  --log_dir qwen3moe

The verification scripts read the existing inference pickle/NumPy files and write self.pkl into the same prompt directories. Use --out_dir for dense models or --log_name for MoE models when a separate result is required.

3. DeepSeek verification

Use a third environment for the older Transformers fork:

cd transformers-4.39.3
python3 -m pip install -e .
python3 -m pip install accelerate datasets tokenizers

Download deepseek-ai/DeepSeek-V2-Lite revision 604d5664dddd88a0433dbae533b7fe9472482de0 and point the script to the local snapshot containing tokenizer.json:

export IMMACULATE_DEEPSEEK_MODEL_PATH=/path/to/DeepSeek-V2-Lite/snapshot
export CUDA_VISIBLE_DEVICES=0

python3 main_deepseek.py --log_dir dpsk

This verifier intentionally uses FP32. The corresponding inference run uses MoE tensor parallelism of degree 1.

Results and analysis

The verification output can be summarized with the scripts under transformer-4.57.3/data_collect/. They use the same IMMACULATE_CACHE_DIR setting:

cd transformer-4.57.3/data_collect
bash collect.sh

The release does not include model weights, downloaded datasets, caches, or generated experiment logs. Pickle files must only be loaded from trusted experiment runs.