Merge branch 'develop'

Author: Etherpad Release Bot

CHANGELOG.md

@@ -1,3 +1,41 @@
+# 3.1.0
+
+3.1 ships the self-update programme's **Tier 4 — autonomous in a maintenance window** for real (the v3.0.0 notes documented the design; this is the release the code actually lands in), adds first-class SMTP delivery so update failures email the admin, and bundles a defence-in-depth pass across the HTTP/API entry points. Two new admin-facing escape hatches arrive: a preflight check that aborts an update *before* it mutates the working tree when the target tag's `engines.node` doesn't match the running runtime, and email notifications for every auto-rollback / preflight outcome (not just the terminal `rollback-failed` state).
+
+### Notable enhancements
+
+- **Self-update — Tier 4 (autonomous in a maintenance window).** Set `updates.tier: "autonomous"` together with `updates.maintenanceWindow: {"start":"HH:MM","end":"HH:MM","tz":"local"|"utc"}` to constrain autonomous updates to a nightly window. The scheduler snaps `scheduledFor` forward to the next window opening when grace would otherwise land outside the window, and defers the fire when the window has closed by the timer callback. Cross-midnight windows (`end < start`) are supported; DST transitions are absorbed by host wall-clock arithmetic. A missing or malformed window degrades the policy to Tier 3 with an explicit `policy.reason` of `maintenance-window-missing` / `maintenance-window-invalid`; an admin banner surfaces the misconfiguration so autonomous behaviour is not silently disabled. The admin update page shows a "Maintenance window" section with the parsed window summary, the next opening, and a "deferred until <iso>" subtitle on the scheduled panel when the timer has been snapped forward. Closes #7607 (#7753).
+- **Updater — real SMTP via nodemailer (new top-level `mail.*` block).** Replaces the "(would send email)" stub. New settings: `mail.host`, `mail.port`, `mail.secure`, `mail.from`, `mail.auth.{user,pass}`. `mail.host=null` keeps the legacy log-only behaviour. The `nodemailer` dependency is lazy-imported on first send so installs that don't configure mail pay no runtime cost; the transport is cached on the full SMTP options tuple so a `reloadSettings()` change to host/port/credentials invalidates the cache. `settings.json.docker` reads `MAIL_HOST` / `MAIL_FROM` / `MAIL_PORT` / `MAIL_SECURE` from env. Send errors are logged warn and swallowed so a transient SMTP failure can never poison the updater state machine.
+- **Updater — preflight against the target tag's `engines.node`.** Before mutating the working tree, `runPreflight` now runs `git show <tag>:package.json` and verifies `process.versions.node` satisfies the target's `engines.node`. A mismatch fails cleanly at `preflight-failed` with the detail `target requires Node >=X, running Y` — no drain, no restart, no rollback. The check runs *after* signature verification so we only trust signed `package.json`. New `PreflightReason: 'node-engine-mismatch'`.
+- **Updater — email admin on rollback / preflight-failed (not just `rollback-failed`).** Before this release only the terminal `rollback-failed` state emailed. Auto-recovered failures (`rolled-back-install-failed`, `rolled-back-build-failed`, `rolled-back-health-check`, `rolled-back-crash-loop`) and `preflight-failed` now also fire one email per `<outcome>:<targetTag>` (dedupe key in `EmailSendLog.lastFailureKey`). A 3am autonomous update that rolls back because of, say, a Node engine bump now lands in the admin inbox at 3am instead of staying invisible until the next admin login. Boot-path catch-up covers cases where the failure preceded a clean process exit (timer-fired health-check rollback, crash-loop forced rollback, preflight-failed that didn't get to email before exit).
+- **API — `listAuthorsOfPad` filters the synthetic system author.** `Pad.SYSTEM_AUTHOR_ID` (`a.etherpad-system`) is the placeholder Etherpad attributes to when the HTTP API receives a call without an `authorId` (setText, setHTML, appendText, server-side import). It was leaking through `listAuthorsOfPad`, making pads with only API-driven content appear to have one "real" author. The synthetic id is now filtered at that API surface only — `getAllAuthors()` and downstream callers (copy, anonymize, atext verification) still see it. Fixes #7785 / #7790 (#7793).
+
+### Notable fixes
+
+- **Export HTML — ordered-list counter no longer poisoned by a sibling unordered list.** When an ordered-list level was the only consumer of `olItemCounts`, closing *any* list at that depth (including a `<ul>` that happened to share the level) reset the counter to 0. A subsequent unrelated `<ol>` at the same depth then took the "counter exists but is 0" branch and emitted `<ol class="...">` without the `start=` attribute. The reset is now gated on `line.listTypeName === 'number'` so closing an unordered list never touches the ol bookkeeping. Fixes #7786 / #7787 (#7791).
+- **Export — bad `:rev` returns a meaningful 500 body, not Express's HTML error page.** A non-numeric `:rev` (e.g. `/p/foo/test1/export/txt`) reached `checkValidRev` which throws `CustomError('rev is not a number', 'apierror')`; the message fell through `.catch(next)` and Express's default renderer returned an HTML 500 page. The route handler now catches the apierror and emits `err.message` as a deterministic `text/plain` 500. As a follow-up, `checkValidRev` runs *before* `res.attachment()` so an invalid rev no longer leaves a `Content-Disposition` header in place (browsers were offering to save the error message as a file), and unrelated export failures (conversion, fs, soffice) are surfaced as text/plain rather than the HTML stack page. Fixes #7788 (#7792).
+
+### Security hardening
+
+A bundle of defence-in-depth tightening picked up during an internal audit pass (#7784):
+
+- **HTTP API — OAuth JWT path.** Verify the signature *before* reading any claim off the payload; require `admin: true` strictly (presence is no longer sufficient). The apikey comparison switches to `crypto.timingSafeEqual`.
+- **Import/Export temp-file path tokens.** Derived from `crypto.randomBytes(16)` instead of `Math.random()`.
+- **Token transfer.** Records now have a 5-minute TTL and are single-use (removed from the store before responding). The author token is no longer in the redemption response body — the `HttpOnly` cookie is the only delivery channel.
+- **`x-proxy-path` header sanitiser (new `src/node/utils/sanitizeProxyPath.ts`).** Shared by `admin.ts` and `specialpages.ts`. Strips characters outside `[A-Za-z0-9_./-]`, collapses leading `//+` to a single `/`, rejects `..` traversal. `admin.ts` also emits `Vary: x-proxy-path` and `Cache-Control: private, no-store` so a poisoned response can never be reused for another origin.
+- **`Pad.appendRevision` insert-op author invariant.** Centralises the "every insert op carries an `author` attribute" rule the socket handler already enforced, so non-wire callers (`setText`, `setHTML`, `restoreRevision`, plugin paths) get the same check. `Pad.init` and `setPadHTML` substitute `SYSTEM_AUTHOR_ID` when no author is supplied — same pattern `setText` / `spliceText` already used.
+- **`setPadRaw` legacy-import rewrite.** Bulk-import bypasses `appendRevision`, so a hand-crafted `.etherpad` file could persist non-conforming records that any subsequent `setText` / `setHTML` would refuse to extend. A pre-pass now walks revs in order, sanitises each changeset's `+` ops against the cumulative pad pool (substituting `SYSTEM_AUTHOR_ID` where needed), and re-applies each changeset to a running atext so the head atext and key-rev `meta.atext` / `meta.pool` snapshots stay in lock-step. Conforming payloads round-trip unchanged.
+
+### Internal / contributor-facing
+
+- **Backend tests — `tests/backend/specs/{api,admin}/*` un-skipped.** The pnpm test script's glob (`tests/backend/specs/**.ts`) only matched depth-1 files. Every spec under `api/` (14 files) and `admin/` (2 files) has been silently skipped by CI. Switched to `--extension ts --recursive` so mocha walks the tree as documented. A new vitest regression check reads the pnpm script, hands mocha the same arguments under `--dry-run --list-files`, and asserts representative specs from both subdirectories appear in the discovered list (#7789).
+- **CI — Windows `npx ENOENT` in the glob-discovery regression check.** `execFileSync('npx', ...)` doesn't pick up `npx.cmd` on Windows runners. Resolved by running `mocha`'s JS entry directly via `require.resolve` under the current node process. Path normalisation now goes through `path.relative` + `replace([\\/])` so mixed-separator / drive-letter casing on Windows mocha output still matches the POSIX-relative assertions (#7794).
+- **CI — `anonymizeAuthorSocket` suite gated on admin-socket health when `ep_hash_auth` is installed.** Un-hiding the suite in #7789 surfaced a 14-minute stall on every with-plugins matrix run because `ep_hash_auth`'s `handleMessage` hook fires for every socket message regardless of namespace and reads from the deprecated `client` context (undefined for non-pad namespaces). Until the root cause lands (tracked in #7795), the suite skips itself when an application-level probe shows the admin `/settings` namespace isn't responding — keeps the no-plugin matrix covered and stops burning ~14 minutes per with-plugins run (#7796).
+
+### Localisation
+
+- Multiple updates from translatewiki.net.
+
 # 3.0.0
 
 3.0 is a feature-heavy release that closes out the self-update programme (Tiers 2 and 3 land alongside Tier 1 from 2.7.3), removes the last identified upstream telemetry vector, and ships a parsed JSONC settings editor, native DOCX export, in-place pad history scrubbing, and an admin UI for GDPR author erasure. It also marks the start of the broader Etherpad app ecosystem (see *Companion apps* below).
@@ -29,7 +67,9 @@ Both clients hit the **stable 3.x API surface**, so server operators don't need
 - **Self-update subsystem — Tier 3 (auto with grace window).**
   - On a git install, set `updates.tier: "auto"` to have new releases applied automatically after `preApplyGraceMinutes`. During the grace window, `/admin/update` shows a live countdown plus Cancel and Apply now buttons. Schedules are persisted to `var/update-state.json`, so an Etherpad restart during the grace window rehydrates the timer instead of losing the schedule. A new release tag detected mid-grace re-arms the timer; if `adminEmail` is set, a one-shot `grace-start` notification fires per scheduled tag (issue #7607).
   - The terminal `rollback-failed` state continues to disable auto/autonomous attempts globally until acknowledged; manual click stays available because an admin click *is* the intervention the terminal state requires.
-  - Tier 4 (autonomous in a maintenance window) remains designed but unimplemented and will land in a subsequent release.
+- **Self-update subsystem — Tier 4 (autonomous in a maintenance window).**
+  - Set `updates.tier: "autonomous"` together with `updates.maintenanceWindow: {"start":"HH:MM","end":"HH:MM","tz":"local"|"utc"}` to constrain autonomous updates to a nightly window. The scheduler snaps `scheduledFor` forward to the next window opening when grace would otherwise land outside the window, and defers the fire when the window has closed by the timer callback. Cross-midnight windows (`end < start`) are supported; DST transitions are absorbed by the host's wall-clock arithmetic.
+  - A missing or malformed window degrades the policy to Tier 3 with an explicit `policy.reason` of `maintenance-window-missing` / `maintenance-window-invalid`; an admin banner surfaces the misconfiguration so autonomous behavior is not silently disabled. Closes #7607.
 - **Privacy — drop swagger-ui telemetry, document phone-homes, add opt-outs.**
   - Dropped `swagger-ui-express` because upstream injects a Scarf analytics pixel that cannot be disabled at install or runtime (see [swagger-api/swagger-ui#10573](https://github.com/swagger-api/swagger-ui/issues/10573)). `/api-docs` now serves a vendored copy of [Scalar](https://github.com/scalar/scalar) (MIT) configured with `withDefaultFonts: false` and `telemetry: false` so no outbound calls are made.
   - New `privacy.updateCheck` (default `true`) — set to `false` to disable the hourly `UpdateCheck.ts` request to `${updateServer}/info.json`.

admin/package.json

@@ -1,7 +1,7 @@
 {
   "name": "admin",
   "private": true,
-  "version": "3.0.0",
+  "version": "3.1.0",
   "type": "module",
   "scripts": {
     "dev": "pnpm gen:api && vite",

admin/src/components/UpdateBanner.tsx

@@ -52,6 +52,23 @@ export const UpdateBanner = () => {
     );
   }
 
+  // Tier 4: tier is autonomous but the maintenance window isn't usable.
+  // Surface that before the generic "update available" banner so the admin
+  // knows the autonomous behavior is sitting idle.
+  const policyReason = updateStatus.policy?.reason;
+  if (updateStatus.tier === 'autonomous'
+      && (policyReason === 'maintenance-window-missing'
+          || policyReason === 'maintenance-window-invalid')) {
+    return (
+      <div className="update-banner update-banner-window" role="status">
+        <strong>
+          <Trans i18nKey={`update.banner.${policyReason}`}/>
+        </strong>{' '}
+        <Link to="/update">{t('update.banner.cta')}</Link>
+      </div>
+    );
+  }
+
   // Tier 3: scheduled update — show countdown banner instead of the plain
   // "update available" one.
   if (updateStatus.execution?.status === 'scheduled') {

admin/src/pages/UpdatePage.tsx

@@ -192,6 +192,54 @@ export const UpdatePage = () => {
               values={{tag: scheduled.targetTag, remaining: fmtRemaining(remainingMs)}}
             />
           </p>
+          {/* Tier 4: only surface the deferral subtitle when `scheduledFor`
+              was actually snapped forward to the next window opening. The
+              backend keeps `scheduledFor = now + grace` whenever that lands
+              inside the window, so we can't use a fixed time-distance
+              heuristic (a normal 15-min grace would falsely match). Instead,
+              compare against `nextWindowOpensAt` with a small tolerance — the
+              two are computed seconds apart at request time, so an exact-ish
+              match is the only safe signal that the schedule was deferred. */}
+          {us.tier === 'autonomous' && us.nextWindowOpensAt
+              && Math.abs(new Date(scheduled.scheduledFor).getTime()
+                          - new Date(us.nextWindowOpensAt).getTime()) < 60 * 1000 && (
+            <p className="update-scheduled-deferred">
+              <Trans
+                i18nKey="update.page.scheduled.deferred_until"
+                values={{at: us.nextWindowOpensAt}}
+              />
+            </p>
+          )}
+        </section>
+      )}
+
+      {us.tier === 'autonomous' && (
+        <section className="update-maintenance-window">
+          <h2><Trans i18nKey="update.window.title"/></h2>
+          {us.maintenanceWindow ? (
+            <>
+              <p>
+                <Trans
+                  i18nKey="update.window.summary"
+                  values={{
+                    start: us.maintenanceWindow.start,
+                    end: us.maintenanceWindow.end,
+                    tz: us.maintenanceWindow.tz,
+                  }}
+                />
+              </p>
+              {us.nextWindowOpensAt && (
+                <p>
+                  <Trans
+                    i18nKey="update.window.next_opens_at"
+                    values={{at: us.nextWindowOpensAt}}
+                  />
+                </p>
+              )}
+            </>
+          ) : (
+            <p><Trans i18nKey="update.window.unset"/></p>
+          )}
         </section>
       )}
 

admin/src/store/store.ts

@@ -25,6 +25,12 @@ export type LastResult = null | {
   at: string;
 };
 
+export interface MaintenanceWindow {
+  start: string;
+  end: string;
+  tz: 'local' | 'utc';
+}
+
 export interface UpdateStatusPayload {
   currentVersion: string;
   latest: null | {
@@ -44,6 +50,9 @@ export interface UpdateStatusPayload {
   execution: Execution;
   lastResult: LastResult;
   lockHeld: boolean;
+  // Tier 4 additions:
+  maintenanceWindow: MaintenanceWindow | null;
+  nextWindowOpensAt: string | null;
 }
 
 type ToastState = {

bin/package.json

@@ -1,6 +1,6 @@
 {
   "name": "bin",
-  "version": "3.0.0",
+  "version": "3.1.0",
   "description": "",
   "main": "checkAllPads.js",
   "directories": {