Huang Xin
9155ae627c
* feat(sync): decouple the incremental-pull cursor from updated_at (#4678) `books.updated_at` was overloaded as both the incremental-pull cursor (`GET /api/sync?since=…` filters `updated_at > since`, devices keep one global `max(updated_at)` watermark) and the library "date read" sort key. A server-resolved reading-status merge had to be written with a timestamp greater than every peer's global cursor to propagate, which forced `updated_at = now()` and reordered the date-read library by sync-processing time (the #4677 symptom). Introduce a server-assigned `synced_at` column on `books`, stamped by a `BEFORE INSERT OR UPDATE` trigger on every write, used only as the pull cursor. `updated_at` stays pure client event time used only for sorting. - Migration 016 + baseline schema: add `synced_at` (NOT NULL DEFAULT now()), index `(user_id, synced_at)`, trigger `set_books_synced_at`. Backfill `synced_at = updated_at` before creating the trigger so existing devices' cursors hand over without a re-sync storm. - GET: books filters/orders on `synced_at > since` (a delete bumps synced_at, so the deleted_at clause is dropped); configs/notes stay on updated_at. - POST: extract `buildStatusPropagationRow` and drop the `updated_at = now()` bump — the trigger advances synced_at so peers re-pull the status change while updated_at (the sort key) stays put. - Client `computeMaxTimestamp` keys on synced_at, falling back to updated_at/deleted_at for pre-migration servers and configs/notes. Backward-compatible: `synced_at >= updated_at` always, so `synced_at > since` is a strict superset of `updated_at > since` — old web clients and the koplugin keep working with no data loss (at worst a redundant idempotent re-pull of rare server-merged rows). The koplugin's shared pull/push cursor is left untouched; a proper split is a follow-up. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(sync): make the books synced_at backfill safe for large live tables (#4678) The single `UPDATE … WHERE synced_at IS NULL` deadlocked on a 3.8M-row production `books` table: it rewrites every row in one transaction while the live /api/sync push path upserts books rows, and the two lock rows in opposite orders. `ALTER COLUMN … SET NOT NULL` (full-table ACCESS EXCLUSIVE scan) and a plain CREATE INDEX (write-blocking SHARE lock) compounded it. Rework migration 016 as an online migration (run via psql, not in a wrapping transaction): - backfill in small autocommitted batches via a procedure, using FOR UPDATE SKIP LOCKED so it never waits on an app-locked row; - CREATE INDEX CONCURRENTLY instead of a blocking build; - install the trigger last (so it can't clobber the updated_at backfill); - drop the hard SET NOT NULL (the default + trigger + backfill keep the column populated and the client falls back to updated_at); a NOT VALID CHECK + VALIDATE alternative is included, commented, for operators who want it. The baseline schema.sql (fresh, empty installs) keeps the simple inline NOT NULL DEFAULT now() + trigger. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
32 lines
1.3 KiB
TypeScript
32 lines
1.3 KiB
TypeScript
import { describe, expect, it } from 'vitest';
|
|
import { computeMaxTimestamp } from '@/hooks/useSync';
|
|
import type { BookDataRecord } from '@/types/book';
|
|
|
|
const iso = (ms: number) => new Date(ms).toISOString();
|
|
|
|
// Issue #4678: the incremental-pull cursor is decoupled from updated_at. The
|
|
// server stamps a `synced_at` on every books write, so the client watermark
|
|
// keys on synced_at and ignores updated_at (the client event time / sort key)
|
|
// and deleted_at (a delete bumps synced_at too).
|
|
describe('computeMaxTimestamp (synced_at pull cursor)', () => {
|
|
it('keys on synced_at, ignoring updated_at and deleted_at', () => {
|
|
const max = computeMaxTimestamp([
|
|
{ synced_at: iso(5000), updated_at: iso(1000), deleted_at: null },
|
|
{ synced_at: iso(3000), updated_at: iso(9999), deleted_at: iso(8000) },
|
|
] as unknown as BookDataRecord[]);
|
|
expect(max).toBe(5000);
|
|
});
|
|
|
|
it('falls back to updated_at/deleted_at when synced_at is absent (old server)', () => {
|
|
const max = computeMaxTimestamp([
|
|
{ updated_at: iso(1000), deleted_at: iso(4000) },
|
|
{ updated_at: iso(2000), deleted_at: null },
|
|
] as unknown as BookDataRecord[]);
|
|
expect(max).toBe(4000);
|
|
});
|
|
|
|
it('returns 0 for empty input', () => {
|
|
expect(computeMaxTimestamp([])).toBe(0);
|
|
});
|
|
});
|