deploy: self-host the PWA behind nginx and systemd

Railway runs app/pit. dev.moshcode.sh is meant to run on a plain VPS
instead, and there was nothing in the repo describing how.

apps/pwa/deploy/ is that kit: a systemd unit, an nginx vhost, an
environment file for a public box, and bootstrap.sh, which does every
step that does not need root and prints the four that do.

The split is deliberate. Root steps are a hand-off on our boxes, so
rather than half-provision and leave you debugging the script,
bootstrap.sh finishes with a smoke test: it boots the app on a scratch
port exactly as systemd will and waits for /healthz. If that passes, the
app is known-good before nginx exists, so every later failure is the
proxy or the certificate.

Details worth keeping:

- The unit is rendered, not hardcoded. It resolves node from
  process.execPath rather than PATH, because a mise shim needs mise's
  environment and systemd has neither it nor the working directory —
  a unit built from the shim dies at 203/EXEC, an error naming no code.
- ReadWritePaths carves out data/ from ProtectHome=read-only. Without
  it SQLite reports "attempt to write a readonly database" at the first
  write rather than at startup.
- No EnvironmentFile. src/config.mjs already loads .env with its own
  parser, and systemd's does not strip the same quotes, so a quoted
  secret would arrive still quoted and fail far from here.
- npm ci, not pnpm: apps/pwa carries its own lockfile and is deployed
  as its own root, so a pnpm install at the repo root does not give
  this directory its dependencies.
- The vhost goes in after certbot issues. nginx refuses to start when
  an ssl_certificate path is missing, so installing it first takes down
  every other site on the box.

The DNS preflight probes a name nobody would create and compares it to
the target. moshcode.sh has a wildcard, which is how dev.moshcode.sh
resolves today while having no record of its own — "it resolves" would
otherwise have green-lit a certbot run that issues for the wrong box.

Verified end to end on a clean copy: npm ci, generated SESSION_SECRET
(32 bytes, mode 600), rendered both files, and the app answered
/healthz with {"ok":true,"env":"production"}. Re-run is idempotent.
systemd-analyze verify passes. 547/547 apps/pwa tests still green.

Author: ralyodio

apps/pwa/.gitignore

@@ -4,5 +4,8 @@ data/
 *.db-*
 .env
 .env.local
+# Rendered by deploy/bootstrap.sh — holds one box's absolute paths, so it is
+# generated per machine rather than committed.
+deploy/out/
 npm-debug.log*
 .DS_Store

apps/pwa/deploy/.env.example

@@ -0,0 +1,83 @@
+# dev.moshcode.sh — the self-hosted dev instance of the PWA.
+#
+# Copy to apps/pwa/.env on the target box. `bootstrap.sh` does that for you and
+# fills SESSION_SECRET with real entropy; everything here is the shape it writes.
+#
+# This is deliberately NOT apps/pwa/.env.example. That one describes a laptop:
+# NODE_ENV=development, localhost, an insecure session secret. A box on the
+# public internet that inherits those settings serves secure-cookie-less
+# sessions signed with a secret published in git, so the two files stay
+# separate rather than one growing comments about when to ignore it.
+
+# `production` is what sets `secure` on the session cookie (see src/config.mjs).
+# It is the environment's posture, not a claim that this is the prod deploy.
+NODE_ENV=production
+
+# Plain HTTP on loopback. nginx holds the certificate and proxies here — see
+# the comment at the top of nginx-vhost.conf for why the app must not own it.
+PORT=8790
+
+# Drives the WebAuthn relying-party ID, which is this hostname.
+#
+# A passkey is bound to the rpID it was registered under, so passkeys created
+# on app.moshcode.sh DO NOT WORK HERE and vice versa. Everyone testing against
+# this box registers a second passkey. That is inherent to WebAuthn, not a
+# misconfiguration to hunt for the first time someone cannot sign in.
+PUBLIC_ORIGIN=https://dev.moshcode.sh
+
+# Where /pit and /n/<name> consider themselves to live. Left equal to
+# PUBLIC_ORIGIN so a name published on this box shows dev URLs; point it at
+# https://pit.moshcode.sh if you want this instance to render pages that
+# advertise prod's namespace instead.
+PIT_ORIGIN=https://dev.moshcode.sh
+
+# datastore.
+#
+# `file:` keeps the dev instance's data on the box and independent of prod —
+# which is the point of a dev environment. Set a Turso URL + token instead if
+# you want it reading a shared database, but note that migrations run at boot
+# (src/migrate.mjs), so pointing a dev instance at prod's database MIGRATES
+# PROD the next time this service restarts.
+DATABASE_URL=file:./data/dev.db
+DATABASE_AUTH_TOKEN=
+
+# bootstrap.sh replaces this with `openssl rand -hex 32`. If you are writing
+# this file by hand, do the same — leaving a literal here signs every session
+# on the box with a value that is in git.
+SESSION_SECRET=
+
+# CLI ↔ app signed approval ingest. Must match the moshcode CLI's
+# MOSHCODE_WEBHOOK_SECRET, or signed ingest 401s with a valid-looking request.
+MOSHCODE_WEBHOOK_SECRET=
+
+# Everything below is optional and inert when blank — the app degrades to "that
+# feature is off" rather than failing to boot. Fill in only what you are
+# actually exercising on this box.
+
+# email via Resend
+RESEND_API_KEY=
+RESEND_FROM=moshcode <notify@moshcoding.com>
+
+# web push (VAPID) — generate with:
+#   node -e "console.log(require('web-push').generateVAPIDKeys())"
+# Use a DIFFERENT keypair from prod. A subscription is bound to the key that
+# created it, so sharing one lets this box push to prod's subscribers.
+VAPID_PUBLIC=
+VAPID_PRIVATE=
+VAPID_SUBJECT=mailto:anthony@profullstack.com
+
+# CoinPay: usage-credit top-ups + "sign in with CoinPay".
+# The redirect URI is derived as PUBLIC_ORIGIN/auth/coinpay/callback, so this
+# host must be registered with CoinPay separately from prod's before the OAuth
+# round-trip will complete.
+COINPAY_API_BASE=https://coinpayportal.com
+COINPAY_BUSINESS_ID=
+COINPAY_WEBHOOK_SECRET=
+COINPAY_OAUTH_AUTHORIZE_URL=
+COINPAY_OAUTH_TOKEN_URL=
+COINPAY_OAUTH_USERINFO_URL=
+COINPAY_OAUTH_CLIENT_ID=
+
+# optional channel providers
+TELEGRAM_BOT_TOKEN=
+SLACK_WEBHOOK_URL=

apps/pwa/deploy/README.md

@@ -0,0 +1,96 @@
+# Running the PWA on your own box
+
+Railway runs `app.moshcode.sh` and `pit.moshcode.sh`. This directory is for the
+other case — a plain VPS with nginx and systemd, which is how `dev.moshcode.sh`
+is meant to run and how anyone self-hosting the app would do it.
+
+| file | what it is |
+|---|---|
+| `bootstrap.sh` | does every step that does not need root, then prints the four that do |
+| `.env.example` | the dev instance's environment — **not** the same as `apps/pwa/.env.example` |
+| `moshcode-dev.service` | systemd unit template |
+| `nginx-vhost.conf` | nginx vhost template |
+| `out/` | where `bootstrap.sh` writes the rendered copies. Git-ignored — it holds this box's paths |
+
+## Before you start
+
+**The DNS record has to exist**, because certbot validates over HTTP-01:
+
+```
+dev  A  <this box's IPv4>
+```
+
+Point it at the machine that will actually serve the app. If that machine is
+IPv6-only, do **not** publish an AAAA-only record for a URL you intend to share
+— it is unreachable from every IPv4-only network. Put the `A` record on a box
+that has IPv4 and reverse-proxy to the v6 machine, the way `dns.moshcode.sh`
+already works.
+
+## The whole thing
+
+```sh
+./apps/pwa/deploy/bootstrap.sh
+```
+
+Then run the four `sudo` commands it prints. That is the entire process.
+
+`bootstrap.sh` is safe to re-run. It will not overwrite an existing `.env`, and
+it regenerates the rendered files every time — so after moving the checkout or
+changing the port, re-run it and reinstall the unit.
+
+Options: `APP_HOST=` `APP_PORT=` `APP_USER=` `NODE_BIN=` as environment
+variables, `--render-only` to emit configs without installing or booting
+anything, `--skip-smoke` to skip the boot test.
+
+## Why it stops before root
+
+Root steps are a hand-off on our boxes, and a provisioning script that
+half-succeeds leaves you debugging the script instead of the service. So
+`bootstrap.sh` takes it as far as it can go unprivileged and finishes with a
+**smoke test**: it boots the app on a scratch port, exactly as systemd will,
+and waits for `/healthz`.
+
+That test is the useful part. If it passes, the app is known-good before nginx
+exists, so every later failure is the proxy or the certificate — and the first
+question when a deploy misbehaves is already answered.
+
+## Order matters in one place
+
+Install the vhost **after** certbot has issued, not before. The vhost
+references `/etc/letsencrypt/live/<host>/fullchain.pem`, and nginx refuses to
+start when an `ssl_certificate` path does not exist — so installing it first
+takes down every other site on the box until you notice. The printed commands
+are in the right order already.
+
+## Things that will look like bugs
+
+- **Your passkeys do not work here.** A passkey is bound to the WebAuthn
+  relying-party ID, which is the hostname. `app.moshcode.sh` and
+  `dev.moshcode.sh` are different rpIDs, so everyone registers a second passkey
+  on the dev box. Inherent to WebAuthn, not something to fix.
+- **Migrations run at boot**, inside the app. A restart is also the upgrade.
+  The corollary is that pointing a dev instance's `DATABASE_URL` at prod's
+  database **migrates prod** on the next restart — `.env.example` defaults to a
+  local file for exactly that reason.
+- **`203/EXEC` from systemd** means the unit's `ExecStart` path is wrong,
+  usually because the unit was copied from another box. Re-run `bootstrap.sh`;
+  it resolves the real `node` binary rather than a mise shim, which is a path
+  that only works with mise's environment loaded.
+- **A large publish batch 413s.** The API takes up to 50 items, but
+  `express.json()` in `src/server.mjs` uses its 100kb default, which a batch of
+  substantial posts exceeds. The vhost allows 2m so nginx is not a second,
+  more confusing limit — the app's own ceiling is the real one.
+
+## Verifying, one layer at a time
+
+Every layer fails identically in a browser, so do not debug from one.
+
+```sh
+sudo systemctl status moshcode-dev        # is it running
+journalctl -u moshcode-dev -n 50 --no-pager
+curl -fsS http://127.0.0.1:8790/healthz   # the app, no nginx involved
+curl -fsS https://dev.moshcode.sh/healthz # the whole path
+```
+
+If the first `curl` passes and the second fails, it is nginx or the
+certificate. If the first fails, stop looking at nginx.