decentralized-gpu-scheduler

This is a project to build an efficient WebGPU allocation system.

RUN WITH:

To-do

• Are there any open source projects that will allow us to run Llama models using WebGPU
• Suggestion is to have one person start handling the front end, and the other person can look into replacing the mock WebGPU models with real WebGPU workers a. Real challenge is setting up these harvested WebGPU models b. Good time to start working on this, Llama is too complicated to run on WebGPU --> Run a very basic model that WebGPU's can run, very run CV or computer vision model or VLM (Vision Language Model) --> FastVLM (released by Apple) c. If we could plug in this model as an example application --> That would be a good goal

Decentralized GPU Function Scheduler

Overview

This project is a web-based framework that demonstrates decentralized GPU scheduling using WebGPU-capable browsers. It consists of three main parts:

1. Frontend (React + Vite) a. Runs in the browser and provides a dashboard for submitting jobs, viewing job history, and visualizing results.
2. Backend (FastAPI + SQLite) a. Handles scheduling logic, job management, and storing users/jobs in a lightweight database. b. Currently runs on a local SQLite database (app.db) which is automatically created.
3. Volunteer Nodes (WebGPU Workers) a. In the future, browsers will volunteer GPU resources to perform computations. b. For now, this piece is stubbed out, but the backend and frontend are already set up to support it

---

decentralized-gpu-scheduler/
│
├── frontend/     # React + Vite frontend
│   └── src/      # React code lives here
│
├── backend/      # FastAPI backend
│   ├── main.py   # Entry point for the server
│   ├── models.py # Database models (User, Job)
│   ├── database.py # Database connection setup
│   └── app.db    # SQLite database file (auto-created)
│
└── README.md     # This file

---

1) Clone the Repository

git clone https://github.com/Rylorx/decentralized-gpu-scheduler.git
cd decentralized-gpu-scheduler

---

2) Frontend Setup (REACT + Vite)

cd frontend
npm install
npm run dev -- --port 5174

---

3) Backend Setup (FastAPI + SQLite)

cd backend
python -m venv venv
.\venv\Scripts\Activate.ps1 # For Windows
pip install -r requirements.txt
uvicorn main:app --reload

---

4) Testing the Backend

• Open your browser at http://127.0.0.1:8000 → should return: {"message": "Backend with SQLite is running"}
• Go to http://127.0.0.1:8000/docs → this opens the Swagger API explorer. a. POST /users/ → create a new user (username + credits). b. GET /users/ → list all users. c. POST /jobs/ → create a new job. d. GET /jobs/ → list all jobs.

---

How It Works

• SQLite database: Simple file-based DB (app.db). Perfect for development because it needs no setup. Later we can switch to Postgres for shared hosting by just changing one connection string.
• Models: Users and Jobs are stored in the DB. Users have credits, jobs have a description/status/result.
• FastAPI: Provides REST API endpoints for the frontend.
• React frontend: Will eventually allow submitting jobs, viewing results, and interacting with the scheduler in a user-friendly way.

Command

• Backend: You have to run this command by activating the .venv in the backend and then running the uvicorn command in root

• python3 -m venv .venv && source .\venv\Scripts\Activate.ps1

• pip install -r requirements.txt

• uvicorn backend.main:app --reload

• Frontend:

• cd frontend

• npm install

• npm run dev -- --port 5174