Museum App

Museum App is a modular, full-stack AI application for detecting, segmenting, and matching visual objects from images. Although designed around a museum use case, the entire pipeline is domain-agnostic and can be easily adapted to other contexts (e.g. retail, industrial inspection, cultural heritage, asset tracking) simply by changing the reference image dataset.

The system combines state-of-the-art computer vision models, SAM3 for object segmentation and DINOv3 for high-level visual understanding, into a unified, production-oriented pipeline that allows users to:

• Detect relevant objects within a scene
• Segment each object with high-precision masks
• Match detected items against a reference dataset

The backend is implemented in Python / FastAPI and exposes dedicated APIs for each stage of the pipeline, while the frontend is built with Next.js, providing an intuitive and basic user experience for exploring results.

At its current stage, the application is intentionally designed to be fully self-contained and in-memory, without relying on external databases. This keeps the system lightweight, easy to deploy, and ideal for experimentation and rapid prototyping, while leaving the architecture open for future integration with persistent storage and production-scale services.

---

Overview

This repository contains a frontend (Next.js) and backend (FastAPI or Python) for the Museum App. This guide explains how to run both services using Docker Compose with proper environment configuration.

Prerequisites

• Docker
• Docker Compose
• HuggingFace account

Envs Configuration

Backend

1. Create a .env file inside /backend You can use .env.example as a reference.
   Add your Hugging Face token with READ permissions (Generate your token here):

   HF_TOKEN="<your-hugging-face-token>"

2. Accept the terms for the required models:

• SAM3
• DINOv3

Frontend

Create .env.local file inside /frontend

cp .env.local.example .env.local

Run

1. Start Docker Desktop / Daemon (if not already running).

2. Build and start containers

   docker compose up --build

3. Access the services

• Frontend: http://localhost:3000
• Backend: http://localhost:8000

---

Running Locally Without Docker

If you want to run frontend and backend locally (without Docker or Docker Compose), you can find detailed instructions in the README files inside the respective folders:

• /backend/README.md
• /frontend/README.md