Skip to content

Repository files navigation

wa-digest

Self-hosted bot that passively reads one or more WhatsApp groups and posts a daily digest (with photos) to a private Telegram chat — one digest per group, separated by a divider. Output language is configurable: Russian by default, with English, Spanish, German and French shipped out of the box.

The bot is read-only on the WhatsApp side: it never sends messages, read receipts, presence, or any group/profile mutation. Participants of the target groups see no sign of its presence. A pytest guard (tests/test_readonly.py) fails the build if any forbidden neonize method is reintroduced in app/collector/.

Full design and rationale: WHATSUP-SUMMURIZER.md.

What it does

Every day at the configured time (default 21:00 Europe/Madrid):

  1. The collector has been quietly storing every message from each target WhatsApp group into SQLite, plus downloading every photo to data/media/.
  2. The digest service feeds the last 24 hours of messages — text plus inline images — to OpenAI gpt-4o-mini (multimodal) and asks for a summary structured by topic. Output language is whatever DIGEST_OUTPUT_LANG is set to.
  3. Each topic in the model's reply carries machine-readable photo refs (📷 <photo_label>: #<id>, where <photo_label> is the localised word for "photo" — Фото / Photos / Fotos / …). The delivery layer strips them from the visible text and posts the matching photos as a Telegram media group right after the summary, then a links section, then a ────────── divider between groups.

Also exposed: manual commands /digest_today, /digest_24h, /links_today, /status (the bot is whitelisted to one Telegram user).

Architecture

Three services from one image, sharing the ./data volume:

  • collector — keeps a WhatsApp session via neonize, filters messages from every configured group, downloads photos to data/media/, writes everything to SQLite. Exposes Kubernetes-style health endpoints: /livez (process alive), /readyz (WhatsApp connected + DB OK), /healthz (alias for /readyz).
  • digest — APScheduler. Daily cron iterates over every configured group, calls OpenAI, parses photo refs, posts per-group summaries to Telegram. A second cron deletes media older than MEDIA_RETENTION_DAYS.
  • telegram_bot — aiogram. Handles the manual commands above. All of them cover every configured group.

Stack

  • Python 3.12 (uv for environments and lockfile).
  • SQLAlchemy 2 async + Alembic. SQLite (aiosqlite) by default; Postgres (asyncpg) supported by changing DATABASE_URL only.
  • neonize (WhatsApp client), aiogram v3 (Telegram), apscheduler (cron), openai v2 SDK (AsyncOpenAI, gpt-4o-mini vision).
  • structlog JSON logs to stdout.

Quick start

# 1. Local prerequisites
brew install uv libmagic            # libmagic is required by neonize at import time
uv sync                             # creates .venv with runtime + dev deps

# 2. Bootstrap (first run on a host)
cp .env.example .env                # then fill in real values for TELEGRAM_*, OPENAI_API_KEY, etc.
docker-compose build
docker-compose run --rm collector alembic upgrade head

# 3. Pair the WhatsApp session
docker-compose run --rm collector python -m app.collector pair
# scan the printed QR code from your phone: WhatsApp → Linked devices

# 4. Discover group JIDs and put them into .env as TARGET_GROUPS
docker-compose run --rm collector python -m app.collector list-groups
# example line: TARGET_GROUPS=120363xxx@g.us:My Group A|120363yyy@g.us:My Group B

# 5. Up
docker-compose up -d
# send /status to the Telegram bot to verify everything is connected

A subtle Docker gotcha: if your install ships only the standalone docker-compose binary (e.g. via Homebrew) and not the Compose CLI plugin, use docker-compose with a hyphen everywhere above, or wire the plugin up once:

mkdir -p ~/.docker/cli-plugins
ln -s "$(which docker-compose)" ~/.docker/cli-plugins/docker-compose

Server deployment (VPS, Raspberry Pi, any 64-bit Linux)

The Quick start works unchanged on a server — only the prerequisite install differs. On a fresh Ubuntu/Debian host:

sudo apt update && sudo apt install -y git docker.io docker-compose
sudo systemctl enable --now docker
sudo usermod -aG docker "$USER"      # then log out and back in

git clone git@github.com:SergeyAP/whatsup-sum.git
cd whatsup-sum
cp .env.example .env
nano .env                            # fill in tokens; keep TARGET_GROUPS empty for now
docker-compose build
docker-compose run --rm collector alembic upgrade head
docker-compose run --rm collector python -m app.collector pair

Pairing over SSH

The QR code is rendered as ASCII art directly in your terminal. Scan it from your phone (WhatsApp → Linked devices). Once you see Successfully paired followed by collector.paired_successfully, press Ctrl+C to exit the pair command.

neonize will likely print a few Got 515 code, reconnecting and Error sending close to websocket warnings between the two events — this is a known whatsmeow post-pairing reconnect quirk, not a failure. The session is already written to data/neonize_session.db by the time you see collector.paired_successfully, and the disconnect can get stuck waiting on a websocket the server has already torn down. Ctrl+C is the intended exit.

Then list groups, copy the JIDs into .env, and start:

docker-compose run --rm collector python -m app.collector list-groups
nano .env                            # TARGET_GROUPS=120363xxx@g.us:Name|...
docker-compose up -d

Supported architectures

neonize ships native binaries only for 64-bit platforms. 32-bit ARM (older Raspberry Pi, armv7l) is not supported — there is no working wheel and the py3-none-any fallback has no Go-side WhatsApp client.

Target Works
x86_64 Linux VPS (KVM, EC2, DigitalOcean, Hetzner, etc.) yes
ARM64 Linux VPS (Hetzner CAX, Oracle Ampere, Scaleway Stardust) yes
Raspberry Pi 4 / 5 with 64-bit OS yes
Raspberry Pi 3 with 64-bit OS yes, but 1 GB RAM is tight
Pi Zero W (1st gen), Pi 1, Pi 2 (all 32-bit) no

Confirm on the target host before deploying:

uname -m         # x86_64 or aarch64 — both OK; armv7l won't work
getconf LONG_BIT # must print 64

Operational essentials

Restart on failure and on reboot. Each service in docker-compose.yml uses restart: unless-stopped. Combined with systemctl enable docker, the stack survives both crashes and host reboots without manual intervention.

Bounded log size. Docker's default JSON log driver grows unbounded. Add to each service in docker-compose.yml:

logging:
  driver: json-file
  options:
    max-size: "10m"
    max-file: "5"

Firewall. All three services are outbound-only (WhatsApp Web, Telegram, OpenAI). The collector's 8080 healthcheck port lives only on the docker bridge. Do not publish ports to the host. On a VPS, ufw allow OpenSSH (or your provider's equivalent) is enough.

Back up the WhatsApp session. data/neonize_session.db is your linked-device credential. If lost, you re-pair from the phone — the historical message store in data/app.db survives, but the collector stops until re-paired. Weekly off-host snapshot:

rsync -a data/neonize_session.db user@offsite:/backup/wa-digest/

Do not sync data/ to an unencrypted public cloud. The session DB is effectively a private key for your WhatsApp device.

Raspberry Pi notes

Only the specifics on top of the generic Linux deployment above:

  • Use a 64-bit image via Raspberry Pi Imager → Raspberry Pi OS (64-bit), or Ubuntu Server 24.04 arm64. The 32-bit image will fail at docker-compose build time when uv sync cannot find a matching wheel.
  • SQLite writes wear out cheap microSD cards over months. Put ./data/ on a USB SSD if you can, or use an endurance-rated card (SanDisk High Endurance, Samsung Pro Endurance).
  • The host timezone does not affect the app (cron timing comes from TIMEZONE in .env, storage is always UTC), but setting it makes reading journalctl and docker logs --timestamps easier:
    sudo timedatectl set-timezone Europe/Madrid

Telegram setup

TELEGRAM_CHAT_ID is the destination chat the bot writes to; TELEGRAM_ALLOWED_USER_ID is the only user allowed to issue commands. For a private DM with the bot the two values are identical — your own Telegram user ID. Get it by writing to @userinfobot.

Before the bot can write to you, you must send it at least one message yourself (Telegram's anti-spam policy). /start is enough.

Languages

Set DIGEST_OUTPUT_LANG in .env to one of the bundled language packs. Both the digest body produced by the LLM and every UI string the bot emits (help text, status, errors, group header, "links today" line) will follow that setting.

Code Language
ru Russian (default)
en English
es Spanish
de German
fr French

Two related variables help the model produce a clean summary:

  • SOURCE_CHAT_LANG — what the source WhatsApp messages are written in. Either auto (the model detects per message) or a free-form name such as Spanish, Spanish and English, Italian.
  • CHAT_TOPIC_HINT — optional one-line domain hint, e.g. automotive chat about Renault models in Barcelona. Leave empty for a generic chat. The hint goes into the LLM system prompt verbatim and encourages the model to capture brands, models, parts and other domain-specific terms.

Adding a new language is a single dataclass entry in app/digest/i18n.py: give it a code, a language_name_en, the three format markers (topics_label, photo_label, links_label) and the UI strings, then add the code to the Literal in app/config.Settings.digest_output_lang.

If the model occasionally drifts away from the configured output language or breaks the 📷 <photo_label>: marker (most common with rare languages or messy mixed-language input), switch OPENAI_MODEL to gpt-4o — it follows the format instructions more strictly.

Environment

Full set in .env.example. The variables that change most often:

Variable Purpose
TARGET_GROUPS Pipe-separated jid:display_name. One digest per entry. Discover JIDs via list-groups; display_name is what appears in the Telegram header.
OPENAI_API_KEY, OPENAI_MODEL OpenAI credentials and model. Defaults to gpt-4o-mini; bump to gpt-4o if the model struggles with your output language.
DIGEST_OUTPUT_LANG Language of the digest body and bot UI. One of ru / en / es / de / fr. Default ru.
SOURCE_CHAT_LANG Hint about the language of incoming WhatsApp messages. auto or a free-form name (e.g. Spanish).
CHAT_TOPIC_HINT Optional domain hint embedded in the LLM system prompt (e.g. automotive chat about Renault). Empty by default.
MAX_IMAGES_PER_DIGEST Hard cap on images sent to the model per digest (default 30, applied per group).
MEDIA_RETENTION_DAYS Daily cleanup deletes files in data/media/ older than this (default 10). Set to 0 to disable.
TELEGRAM_BOT_TOKEN, TELEGRAM_CHAT_ID, TELEGRAM_ALLOWED_USER_ID Telegram destination + whitelist. See section above.
DIGEST_CRON_HOUR, DIGEST_CRON_MINUTE, TIMEZONE When the digest cron fires. Same time for all groups.
COLLECTOR_HEALTHCHECK_URL How telegram_bot reaches collector. Defaults to the compose service name. For local non-docker runs, set to http://127.0.0.1:8080/readyz.
DATABASE_URL SQLite by default; switch to Postgres by changing this URL only.

Bot commands

All of them are restricted to TELEGRAM_ALLOWED_USER_ID:

Command Effect
/start Shows the command menu.
/status Current state: WhatsApp connection, DB backend, per-group message counts for today and last digest timestamps.
/digest_today Generate and send the digest for everything since 00:00 local time today.
/digest_24h Same, but for the last rolling 24 hours.
/links_today Plain list of links posted today (no LLM call, no cost).

Verification gate

Before declaring a change done, all three must be green:

uv run ruff check . && uv run mypy app && uv run pytest -q

Tests cover the read-only WhatsApp contract, the digest parser, the links extractor, the time helpers, the message repository, and the collector noise filter.

Risks

neonize uses the unofficial WhatsApp Web protocol. The paired WhatsApp number can in theory be banned. Run on an account you can afford to lose.

The bot pays OpenAI per digest. With gpt-4o-mini and the default MAX_IMAGES_PER_DIGEST=30 a typical day costs well under a cent, but note that the cron will keep firing whether or not anyone reads the output — make sure TELEGRAM_CHAT_ID is set to a chat that exists before leaving it running unattended.

About

Self-hosted bot that turns WhatsApp group chatter into a daily multilingual Telegram digest with photos.

Resources

Stars

Watchers

Forks

Used by

Contributors

Languages