forked from akai/readest
Huang Xin
9155ae627c
* feat(sync): decouple the incremental-pull cursor from updated_at (#4678) `books.updated_at` was overloaded as both the incremental-pull cursor (`GET /api/sync?since=…` filters `updated_at > since`, devices keep one global `max(updated_at)` watermark) and the library "date read" sort key. A server-resolved reading-status merge had to be written with a timestamp greater than every peer's global cursor to propagate, which forced `updated_at = now()` and reordered the date-read library by sync-processing time (the #4677 symptom). Introduce a server-assigned `synced_at` column on `books`, stamped by a `BEFORE INSERT OR UPDATE` trigger on every write, used only as the pull cursor. `updated_at` stays pure client event time used only for sorting. - Migration 016 + baseline schema: add `synced_at` (NOT NULL DEFAULT now()), index `(user_id, synced_at)`, trigger `set_books_synced_at`. Backfill `synced_at = updated_at` before creating the trigger so existing devices' cursors hand over without a re-sync storm. - GET: books filters/orders on `synced_at > since` (a delete bumps synced_at, so the deleted_at clause is dropped); configs/notes stay on updated_at. - POST: extract `buildStatusPropagationRow` and drop the `updated_at = now()` bump — the trigger advances synced_at so peers re-pull the status change while updated_at (the sort key) stays put. - Client `computeMaxTimestamp` keys on synced_at, falling back to updated_at/deleted_at for pre-migration servers and configs/notes. Backward-compatible: `synced_at >= updated_at` always, so `synced_at > since` is a strict superset of `updated_at > since` — old web clients and the koplugin keep working with no data loss (at worst a redundant idempotent re-pull of rare server-merged rows). The koplugin's shared pull/push cursor is left untouched; a proper split is a follow-up. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(sync): make the books synced_at backfill safe for large live tables (#4678) The single `UPDATE … WHERE synced_at IS NULL` deadlocked on a 3.8M-row production `books` table: it rewrites every row in one transaction while the live /api/sync push path upserts books rows, and the two lock rows in opposite orders. `ALTER COLUMN … SET NOT NULL` (full-table ACCESS EXCLUSIVE scan) and a plain CREATE INDEX (write-blocking SHARE lock) compounded it. Rework migration 016 as an online migration (run via psql, not in a wrapping transaction): - backfill in small autocommitted batches via a procedure, using FOR UPDATE SKIP LOCKED so it never waits on an app-locked row; - CREATE INDEX CONCURRENTLY instead of a blocking build; - install the trigger last (so it can't clobber the updated_at backfill); - drop the hard SET NOT NULL (the default + trigger + backfill keep the column populated and the client falls back to updated_at); a NOT VALID CHECK + VALIDATE alternative is included, commented, for operators who want it. The baseline schema.sql (fresh, empty installs) keeps the simple inline NOT NULL DEFAULT now() + trigger. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>