DeepLook

This project provides a robust and efficient system for accessing historical and real-time trading data from the DeepBook V3 decentralized orderbook on Sui. Built with a focus on performance, reliability, and observability, it delivers OHLCV data, live orderbooks, and trade-level summaries via a simple HTTP API.

---

Features

• 📊 OHLCV Aggregation
  Aggregated candlestick data (Open, High, Low, Close, Volume) at 1-minute resolution using TimescaleDB.

• 🧾 Trade-Level Summaries
  Exposes normalized historical trades (excluding tick-by-tick) per pool with optional time filters.

• 📚 Historical Snapshots
  Live and historical orderbook snapshots per trading pair with timestamped updates.

• 🧩 API Extensibility
  Fully compatible with all endpoints from deepbookv3, with custom endpoints added.

• ⚙️ Monitoring and Metrics
  Includes a full observability stack:

  • Prometheus for metrics
  • Grafana dashboards for visual monitoring

• 🧩 Dashboard
  Dashboard showing important data is live at deeplook.carmine.finance.

---

Architecture

• Backend Framework: Built on top of deepbookv3 fork.
• Database: PostgreSQL + TimescaleDB for high-performance time-series aggregation.
• Deployment: Dockerized services with NGINX reverse proxy and HTTPS enabled.
• Monitoring Stack: Prometheus + Grafana with custom metrics for ingestion and uptime tracking.

---

API Endpoints

All endpoints return JSON and are publicly accessible via HTTPS.

/get_pools

Returns metadata for all available pools.
Example

/ohlcv/<pool_name>?start_time=<unix_sec>&end_time=<unix_sec>&timeframe=<1m|15m|1h|4h>

Returns OHLCV candlestick data for the specified time range and timeframe. Timeframe defaults to 1 min.
Example

/orderbook/<pool_name>

Returns the current orderbook snapshot and the timestamp of the last update.
Example

/order_fills/<pool_name>?start_time=<unix_sec>&end_time=<unix_sec>

Returns all trade-level order fills within the specified time window.
Example

---

WebSocket endpoints

/ws_orderbook/<pool_name>

Returns whole orderbook snapshot via websocket that updates everytime a relevant event happens.

• Example: wss://api.sui.carmine.finance/ws_orderbook/SUI_USDC

/ws_orderbook_bests/:pool_names

Returns current best levels via websocket on every orderbook update (even if the update doesn't happen on best levels).

• Example: wss://api.sui.carmine.finance/ws_orderbook_bests/SUI_USDC

/ws_orderbook_spread/:pool_name

Returns current spread via websocket on every orderbook update (even if the update doesn't happen on best levels).

• Example: wss://api.sui.carmine.finance/ws_orderbook_spread/SUI_USDC

/latest_trades/:pool_name

Returns latest 100 trades every time a new trade is observed.

• Example wss://api.sui.carmine.finance/latest_trades/SUI_USDC

---

Aggregation Endpoints

Endpoints that aggregate data to provide further insights.

/aggregation/avg_trade_size/<pool_name>?start_time=<unix_sec>&end_time=<unix_sec>

Returns the average base and quote volume per trade.
Example

/average_trade_multi_window/:pool_name

Returns the average volume per trade in following windows: [5min, 15min, 1h, 24h] Example

/volume/:pool_name

Returns the total volume in last n-days. Example

/volume_multi_window/:pool_name

Returns the total volume in following windows: [5min, 15min, 1h, 24h] Example

/aggregation/avg_duration_between_trades/<pool_name>?start_time=<unix_sec>&end_time=<unix_sec>

Returns the average time in milliseconds between consecutive trades.
Example

/aggregation/vwap/<pool_name>?start_time=<unix_sec>&end_time=<unix_sec>

Returns the Volume-Weighted Average Price (VWAP) over the selected time window.
Example

/aggregation/orderbook_imbalance/<pool_name>?depth=<int>&level=<1|2>

Returns the normalized order book imbalance (0–100 scale) at a given depth and level.
Example

/fills_24h_summary

Returns trading summary of all pools in the last 24 hours. Example

---

Monitoring

• API and indexer prometheus metrics are gathered
• There is a Grafana dashboard for API and for indexer

Dashboards are available here.

---

Data Storage

• Aggregated OHLCV and trade data is stored in Postgresql with TimescaleDB extension for fast querying.

---

Setup & Deployment

Local development

The project has a Makefile that streamlines local development.

Create .env file for local development in the root directory with the following values:

POSTGRES_USER=postgres
POSTGRES_PASSWORD=postgres
POSTGRES_DB=deeplook
POSTGRES_HOST=localhost
DATABASE_URL=postgres://postgres:postgres@localhost/deeplook
REMOTE_STORE_URL=https://checkpoints.mainnet.sui.io
RPC_URL=https://fullnode.mainnet.sui.io:443
ENV=mainnet
REDIS_URL=redis://localhost:6379

Do not use these values in production

Create Postgresql database locally using Docker container.

make postgres

It is currently required to install Timescaledb manually. Access the container:

docker exec -it deeplook-db /bin/bash

And following Install TimescaleDB on Linux tutorial.

Then create db with

make createdb

Run database migrations

make migrateup

Create Redis locally using Docker container. Redis is used to hold live orderbook.

make redis

Run indexer (currently it runs from checkpoint 150000000 with skip-watermark, feel free to adjust this in the Makefile):

make indexer

Run API

make api

Run orderbook service that keeps track of orderbook events and updates live orderbook in Redis.

make orderbook

Production

It is advised to build docker images from docker folder and use those in production.

You need to have postgresql database with timescaledb extension, then provide ENV variables, specified above, with correct values and run the containers.