---

What Is Leorch?

Leorch is a deep learning framework written in Rust. Not a wrapper around PyTorch. Not bindings to libtorch. An actual ground-up implementation of tensors, autograd, layers, optimizers — the whole stack — in Rust.

If you've ever used PyTorch and wondered what's happening under the hood, this codebase is a good place to look. The architecture is intentionally close to PyTorch's, so concepts map across cleanly.

Built by LEGION CODER DEMO AND HEXA — Death Legion Team LK.

---

Features

Tensors	N-dimensional arrays, math ops, shape ops, indexing
Autograd	Computational graph, backward pass, gradient tracking
Layers	Linear, Conv2D, BatchNorm, LayerNorm, Dropout, MaxPool2d, AvgPool2d
Activations	ReLU, LeakyReLU, Sigmoid, Tanh, GELU, ELU, SELU, Softmax
Loss Functions	MSE, CrossEntropy, BCE, BCEWithLogits, L1, SmoothL1
Optimizers	SGD, Adam, AdamW, RMSprop — all with momentum + weight decay
LR Schedulers	StepLR, MultiStepLR, ExponentialLR, CosineAnnealingLR, ReduceLROnPlateau
Data Loading	Dataset abstractions, DataLoader with batching and shuffling
Serialization	Save and load tensors via serde

---

Installation

Add to your Cargo.toml:

[dependencies]
leorch = { path = "path/to/leorch" }

Or clone and build:

git clone https://github.com/deathlegionteamlk/leorch.git
cd leorch
cargo build --release

---

Quick Start

30-second demo — create tensors, run a linear layer, compute loss.

use leorch::tensor::Tensor;
use leorch::nn::{Module, Linear};
use leorch::loss::{Loss, MSELoss};

fn main() {
    let x = Tensor::randn(&[32, 10]);
    let y = Tensor::randn(&[32, 1]);

    let model = Linear::new(10, 1);
    let output = model.forward(&x);

    let criterion = MSELoss::new();
    let loss = criterion.forward(&output, &y);

    println!("Loss: {}", loss.to_vec()[0]);
}

---

Examples

---

Example 1 — XOR Problem

The classic test for a neural net that can't be solved by a straight line. Two inputs, one output, four data points.

🔍 Click to expand full source

use leorch::tensor::Tensor;
use leorch::nn::{Module, Linear, ReLU, Sigmoid};
use leorch::loss::{Loss, MSELoss};

fn main() {
    let x = Tensor::from_slice(&[
        0.0, 0.0,
        0.0, 1.0,
        1.0, 0.0,
        1.0, 1.0,
    ], &[4, 2]).unwrap();

    let y = Tensor::from_slice(&[0.0, 1.0, 1.0, 0.0], &[4, 1]).unwrap();

    let hidden    = Linear::new(2, 4);
    let activation = ReLU::new();
    let output    = Linear::new(4, 1);
    let sigmoid   = Sigmoid::new();

    let h    = activation.forward(&hidden.forward(&x));
    let pred = sigmoid.forward(&output.forward(&h));

    println!("Predictions: {:?}", pred.to_vec());
}

What the network looks like:

Input (2)  ──►  Hidden ReLU (4)  ──►  Output Sigmoid (1)
  [0,0]                                     [0]
  [0,1]    ──►       2→4→1        ──►       [1]
  [1,0]                                     [1]
  [1,1]                                     [0]

cargo run --example xor

---

Example 2 — Linear Regression

Simplest possible supervised learning. Fit a line to noisy data using MSE loss and SGD.

🔍 Click to expand full source

use leorch::tensor::Tensor;
use leorch::nn::{Module, Linear};
use leorch::loss::{Loss, MSELoss};
use leorch::optim::SGD;

fn main() {
    let x = Tensor::randn(&[64, 1]);
    let y = Tensor::randn(&[64, 1]);

    let model     = Linear::new(1, 1);
    let criterion = MSELoss::new();
    let mut optim = SGD::new(model.parameters(), 0.01);

    for epoch in 0..100 {
        let pred = model.forward(&x);
        let loss = criterion.forward(&pred, &y);

        optim.zero_grad();
        loss.backward();
        optim.step();

        if epoch % 10 == 0 {
            println!("Epoch {epoch}: loss = {:.4}", loss.to_vec()[0]);
        }
    }
}

cargo run --example linear_regression

---

Example 3 — Multi-Layer Classifier

Stack a few layers, throw in BatchNorm and Dropout, train on synthetic class data.

🔍 Click to expand full source

use leorch::tensor::Tensor;
use leorch::nn::{Module, Linear, ReLU, BatchNorm1d, Dropout};
use leorch::loss::{Loss, CrossEntropyLoss};
use leorch::optim::Adam;

fn main() {
    let x = Tensor::randn(&[128, 20]);
    let y = Tensor::randint(0, 5, &[128]);

    let fc1      = Linear::new(20, 64);
    let bn1      = BatchNorm1d::new(64);
    let relu     = ReLU::new();
    let dropout  = Dropout::new(0.3);
    let fc2      = Linear::new(64, 5);
    let criterion = CrossEntropyLoss::new();

    let mut params = vec![];
    params.extend(fc1.parameters());
    params.extend(fc2.parameters());

    let mut optim = Adam::new(params, 0.001);

    for epoch in 0..50 {
        let h    = dropout.forward(&relu.forward(&bn1.forward(&fc1.forward(&x))));
        let logits = fc2.forward(&h);
        let loss = criterion.forward(&logits, &y);

        optim.zero_grad();
        loss.backward();
        optim.step();

        if epoch % 10 == 0 {
            println!("Epoch {epoch}: loss = {:.4}", loss.to_vec()[0]);
        }
    }
}

cargo run --example classifier

---

Example 4 — Conv2D Image Feature Extraction

A small conv stack on a random 3-channel image tensor. Tests Conv2d → BatchNorm2d → ReLU → MaxPool2d.

🔍 Click to expand full source

use leorch::tensor::Tensor;
use leorch::nn::{Module, Conv2d, BatchNorm2d, ReLU, MaxPool2d, Flatten, Linear};
use leorch::loss::{Loss, CrossEntropyLoss};

fn main() {
    let x = Tensor::randn(&[8, 3, 32, 32]);
    let y = Tensor::randint(0, 10, &[8]);

    let conv1   = Conv2d::new(3, 16, (3, 3));
    let bn1     = BatchNorm2d::new(16);
    let relu    = ReLU::new();
    let pool    = MaxPool2d::new((2, 2));
    let flatten = Flatten::new();
    let fc      = Linear::new(16 * 15 * 15, 10);
    let criterion = CrossEntropyLoss::new();

    let h      = pool.forward(&relu.forward(&bn1.forward(&conv1.forward(&x))));
    let flat   = flatten.forward(&h);
    let logits = fc.forward(&flat);
    let loss   = criterion.forward(&logits, &y);

    println!("Conv stack loss: {:.4}", loss.to_vec()[0]);
}

cargo run --example conv_features

---

Example 5 — DataLoader Training Loop

Load batches from a TensorDataset, shuffle each epoch, full gradient loop.

🔍 Click to expand full source

use leorch::tensor::Tensor;
use leorch::nn::{Module, Linear};
use leorch::loss::{Loss, MSELoss};
use leorch::optim::AdamW;
use leorch::data::{TensorDataset, DataLoader};

fn main() {
    let data    = Tensor::randn(&[200, 16]);
    let targets = Tensor::randn(&[200, 1]);
    let dataset = TensorDataset::new(data, targets).unwrap();

    let mut loader = DataLoader::new(dataset, 32);
    loader.shuffle();

    let model     = Linear::new(16, 1);
    let criterion = MSELoss::new();
    let mut optim = AdamW::new(model.parameters(), 0.001);

    for epoch in 0..10 {
        let mut total_loss = 0.0_f32;
        for (batch_x, batch_y) in &mut loader {
            let pred = model.forward(&batch_x);
            let loss = criterion.forward(&pred, &batch_y);

            optim.zero_grad();
            loss.backward();
            optim.step();

            total_loss += loss.to_vec()[0];
        }
        println!("Epoch {epoch}: avg loss = {:.4}", total_loss / 200.0);
    }
}

cargo run --example dataloader_loop

---

API Reference

Tensor Operations

📦 Creation, shape ops, math — click to expand

let zeros     = Tensor::zeros(&[2, 3]);
let ones      = Tensor::ones(&[2, 3]);
let random    = Tensor::randn(&[2, 3]);
let from_data = Tensor::from_slice(&[1.0, 2.0, 3.0], &[3]).unwrap();

let reshaped   = tensor.reshape(&[3, 2]).unwrap();
let flattened  = tensor.flatten();
let transposed = tensor.transpose(0, 1).unwrap();

let sum  = tensor.sum();
let mean = tensor.mean();
let max  = tensor.max();
let min  = tensor.min();

let sqrt = tensor.sqrt();
let exp  = tensor.exp();
let log  = tensor.log();
let pow  = tensor.pow(2.0);

let product = a.matmul(&b).unwrap();

---

Neural Network Layers

🧱 Linear, Conv2d, BN, Dropout, Pooling — click to expand

use leorch::nn::{Linear, Conv2d, BatchNorm2d, Dropout, MaxPool2d, Flatten};

let linear = Linear::new(784, 128);

let conv = Conv2d::new(3, 64, (3, 3));
let conv_with_params = Conv2d::new_with_params(
    3, 64, (3, 3), (1, 1), (1, 1), (1, 1), true,
);

let bn      = BatchNorm2d::new(64);
let ln      = LayerNorm::new(vec![128]);
let dropout = Dropout::new(0.5);

let maxpool = MaxPool2d::new((2, 2));
let avgpool = AvgPool2d::new((2, 2));
let flatten = Flatten::new();

---

Activation Functions

⚡ ReLU, GELU, Sigmoid, Tanh and more — click to expand

use leorch::nn::{ReLU, LeakyReLU, Sigmoid, Tanh, Softmax, GELU, ELU, SELU};

let relu       = ReLU::new();
let leaky_relu = LeakyReLU::with_slope(0.01);
let sigmoid    = Sigmoid::new();
let tanh       = Tanh::new();
let softmax    = Softmax::new(-1);
let gelu       = GELU::new();
let elu        = ELU::with_alpha(1.0);
let selu       = SELU::new();

---

Loss Functions

📉 MSE, CrossEntropy, BCE and more — click to expand

use leorch::loss::{MSELoss, L1Loss, CrossEntropyLoss, BCELoss, BCEWithLogitsLoss};

let mse        = MSELoss::new();
let l1         = L1Loss::new();
let ce         = CrossEntropyLoss::new();
let bce        = BCELoss::new();
let bce_logits = BCEWithLogitsLoss::new();

let mse_sum = MSELoss::with_reduction(Reduction::Sum);

---

Optimizers

🚀 SGD, Adam, AdamW, RMSprop — click to expand

use leorch::optim::{SGD, Adam, AdamW, RMSprop};

let sgd          = SGD::new(params, 0.01);
let sgd_momentum = SGD::with_momentum(params, 0.01, 0.9);

let adam        = Adam::new(params, 0.001);
let adam_custom = Adam::with_betas(params, 0.001, (0.9, 0.999));

let adamw   = AdamW::new(params, 0.001);
let rmsprop = RMSprop::new(params, 0.01);

---

Learning Rate Schedulers

📊 StepLR, Cosine, Exponential — click to expand

use leorch::optim::{StepLR, MultiStepLR, ExponentialLR, CosineAnnealingLR};

let step_lr    = StepLR::new(0.1, 10, 0.1);
let multi_step = MultiStepLR::new(0.1, vec![30, 60, 90], 0.1);
let exp_lr     = ExponentialLR::new(0.1, 0.95);
let cosine     = CosineAnnealingLR::new(0.1, 100);

---

Data Loading

🗂️ TensorDataset, DataLoader, xor_dataset — click to expand

use leorch::data::{TensorDataset, DataLoader, xor_dataset};

let data    = Tensor::randn(&[100, 10]);
let targets = Tensor::randn(&[100, 1]);
let dataset = TensorDataset::new(data, targets).unwrap();

let mut loader = DataLoader::new(dataset, 32);
loader.shuffle();

for (batch_data, batch_targets) in &mut loader {
    // training loop
}

let xor = xor_dataset();

---

Project Structure

leorch/
├── Cargo.toml
├── src/
│   ├── lib.rs
│   ├── tensor.rs
│   ├── autograd.rs
│   ├── nn/
│   │   ├── mod.rs
│   │   ├── layers.rs
│   │   └── activations.rs
│   ├── optim.rs
│   ├── loss.rs
│   ├── functional.rs
│   ├── data.rs
│   └── error.rs
├── examples/
│   ├── xor.rs
│   ├── linear_regression.rs
│   ├── classifier.rs
│   ├── conv_features.rs
│   └── dataloader_loop.rs
└── tests/
    └── integration_tests.rs

---

Architecture

Five layers, each self-contained, each doing exactly one thing:

Layer	Built on	What it does
Tensor Core	ndarray	N-dimensional arrays, all math ops
Autograd	Custom graph	Tracks ops, runs backward pass
NN Module	Trait system	Extensible layer API
Optimizers	Stateful structs	Momentum, adaptive rates
Data	Iterator protocol	Batching, shuffling, datasets

---

Testing

cargo test

cargo test -- --nocapture

cargo test test_tensor_creation

---

Building

cargo build

cargo build --release

cargo run --example xor

---

Performance

CPU ops run in parallel via rayon. Memory layout matches ndarray conventions, so batch processing doesn't fight the allocator. GPU isn't there yet — CUDA bindings are on the roadmap.

---

Roadmap

• GPU support via CUDA
• Recurrent and attention layers
• Save and load entire models
• FP16 / mixed precision
• Distributed training
• Expanded loss and metric library

---

Contributing

Pull requests are welcome. Open an issue first if you're planning something substantial — saves everyone time if we talk through the design before you write the code.

---

Acknowledgments

Built on ndarray for tensor ops, rayon for parallelism, and serde for serialization. The API design follows PyTorch closely — if you know PyTorch, you'll find your way around here quickly.

---

License

MIT — see LICENSE.

---