Tech Stack & Environment

🧰 Tech Stack & Environment

Recommended baseline

• OS: Linux / macOS / WSL2 (Windows native is doable but more painful for bio libs)
• Python: 3.12 (best compatibility across scanpy/anndata/torch stacks)
• Package manager: uv , Installation see : https://docs.astral.sh/uv/getting-started/installation/
• Compute: CPU is fine for Milestone 2. GPU is for Milestone 3.

Core libraries (Milestone 2)

Data + scientific

• numpy, pandas, scipy
• pyarrow (parquet)
• matplotlib

Single-cell / spatial

• anndata
• scanpy
• squidpy (spatial utilities; optional but helpful)

General

• hydra for managing configs
• tqdm
• pydantic (optional, great for schema validation)
• rich (optional, nicer logs)

ML stack (mostly Milestone 3, but OK to install now)

• torch
• pytorch-lightning
• transformers
• datasets
• accelerate
• peft (if doing parameter-efficient tuning)

Dev tooling

• ty (lint) [Strongly recommended] [For installation see: https://docs.astral.sh/ty/installation/]
• black (format) [Strongly recommended] Alternative: Use ruff but requires more setup.
• pre-commit

---

Environment Setup

uv init
uv venv
uv sync

This will create a pyproject.toml for tracking dependencies. For adding a dependency, instead of doing a pip install [dep], do uv add [dep]. This will automatically install the dependency and get listed in the [depedencies] in the pyproject.toml

---

Also: “Docs format so anyone can structure code”

Adopt a strict rule: every pipeline step produces artifacts + a companion doc.

Example mapping:

• notebooks/preprocessing.ipynb → docs/preprocessing_notes.md
• notebooks/partitioning.ipynb → docs/partitioning_strategy.md
• notebooks/stats.ipynb → docs/client_stats_summary.md

And each doc starts with:

## Depends On
- docs/data_dictionary.md
- data/processed/genes.txt
- data/processed/label_map.json