6e9faaa8745fe5db5def8a993ac0220688e19e33
25 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Huang Xin
|
744d5b3a03 |
fix(dict): restore MDD eager init; CSP-safe audio handler; gate binary uploads on sync category (#4337)
* fix(dict): restore MDD eager init; CSP-safe Vocabulary.com audio handler
Two MDict fixes:
1. **Regression**: #4334 turned on `MDD.create(..., { lazy: true })` so
the audio / resource side of every MDict would skip the upfront
key-block decode + sort. That's the right trade-off on the MDX side
(millions of headwords, ~80 s init saving), but it doesn't transfer
to MDDs: js-mdict's lazy lookup binary-searches `keyInfoList`
envelopes with `comp` (localeCompare), and MDDs store keys in
byte-sorted order with `\` prefixes and case folding. The two orders
disagree just often enough that `mdd.locateBytes('sound.png')`
misses on resource paths that do exist — the icon's CSS
`background-image: url(sound.png)` then stays unresolved and the
speaker glyph disappears. Bisected to
|
||
|
loveheaven
|
64651a65ef |
fix(sync): skip replica upload when not authenticated (#4327)
Prevents 'Please log in to continue' error when importing fonts or images locally without logging in. The replica upload is now gated on getAccessToken() so unauthenticated users are never enqueued. |
||
|
Huang Xin
|
5e366018df |
fix(cbz): ComicInfo metadata + CBZ page count + WebDAV i18n (#4282)
* fix(cbz,i18n): ComicInfo metadata + CBZ page count + WebDAV i18n Closes #4253 (ComicInfo.xml not read) and #4255 (CBZ shows "1 page left"). CBZ / ComicInfo (foliate-js submodule + Readest derivation): - comic-book.js: find ComicInfo.xml in subdirectories too, parse description / subject / identifier / published / series fields beyond the prior name+position pair. Series Count populates the canonical `belongsTo.series.total`; no top-level duplication. - bookService.ts / readerStore.ts: derive `metadata.seriesTotal` from `belongsTo.series.total` in parallel to the existing series / seriesIndex derivation. - ProgressBar / FooterBar / DesktopFooterBar: drop the hard-coded `pagesLeft = 1` for fixed-layout books and compute it from `section.total - section.current`. FooterBar uses `FIXED_LAYOUT_FORMATS.has(bookFormat)` so CBZ picks `section` (correct image count) instead of `pageinfo` (locations). - ProgressBar: switch the remaining-pages text to "in book" for fixed-layout titles (no chapter structure) and keep "in chapter" for reflowable books. WebDAV refactor for translation coverage: - WebDAVBrowsePane / SyncHistoryPanel called `t(...)` (passed as a prop) instead of `_(...)`. The i18next-scanner only looks for `_`, so ~53 strings were unreachable and shipped in English to every locale. Switched both components to call `useTranslation()` themselves; helpers that aren't React FCs take `_: TranslationFunc` so the scanner sees the literal calls. - WebDAVClient.checkConnection now returns a `code` discriminator (`SERVER_URL_REQUIRED` / `AUTH_FAILED` / `ROOT_NOT_FOUND` / `UNEXPECTED_STATUS` / `NETWORK`); raw English `message` is reserved for the dev console. New `formatConnectError` and `formatSyncError` helpers in WebDAVForm translate via a switch where each branch is a literal `_('...')`. Same treatment for the sync-failure path that previously surfaced raw e.message. - "Syncing 0 / {{total}}" is now parameterized as "Syncing {{n}} / {{total}}" with n=0 at startup so the digit formats naturally and the template can be reused mid-sync. - "Cleanup · {{count}} book(s)" hard-coded options used unsupported ternary; rewrote as plural-aware key. i18n scanner fix (i18next-scanner.config.cjs): - vinyl-fs walked into directories whose names end in source-file extensions (Next.js route folder `runtime-config.js/`, Playwright screenshot folder `*.test.tsx/`) and crashed with EISDIR. Resolved by expanding globs via `fs.globSync` and filtering to files only before handing to the scanner. TypeScript-syntax sites that broke esprima during extraction: - WebDAVBrowsePane / WebDAVForm: `(e as Error).message` and `failed[0]!.title` inside `_(..., options)` arguments. Replaced with `e instanceof Error ? e.message : String(e)` and `failed[0]?.title ?? ''` — also runtime-safer. User-facing em-dash cleanup: - Removed em-dashes from translation keys across SyncHistoryPanel / WebDAVForm / WebDAVBrowsePane / SyncPassphraseSection / send/page / replicaCryptoMiddleware / AIPanel. Tagline in `layout.tsx` kept. Locale translations: - ~2400 translations applied across all 33 locales for the keys that were either newly extractable, freshly worded, or pre-existing but untranslated. Zero `__STRING_NOT_TRANSLATED__` remain after the run. Misc: - next.config.mjs: drop `eslint.ignoreDuringBuilds: true` so build runs the same lint as CI. - Collection type: add `total?: string` for ComicInfo series count. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * ci(test): fix vitest invocation, run with 4 workers `pnpm test:pr:web` was chaining `pnpm test -- --watch=false`, which pnpm expanded into: dotenv -e .env -e .env.test.local -- vitest -- --watch=false The second `--` made vitest treat `--watch=false` as a positional file pattern, not a flag. Vitest then fell back to defaults (in CI's non-TTY env that still meant a one-shot run, so the suite passed), but the worker pool was effectively serialized for big chunks of the 243-file run — wall ~90 s on a 4-vCPU runner where the parallel-sum of phases was ~236 s (≈2.6× effective parallelism). Replace the chained pnpm invocation with a direct call to `vitest run --maxWorkers=4`, matching the 4 vCPUs the GH Actions ubuntu-latest runner provides. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
a20f68fc11 |
feat(readwise): allow overriding the Readwise sync base URL (#4196)
* feat(readwise): allow overriding the Readwise sync base URL Add an advanced option to point Readwise sync/export at a custom, Readwise-compatible endpoint instead of the hardcoded official API. When the override is unset or blank, behavior is unchanged. - ReadwiseClient resolves a custom `baseUrl` over `READWISE_API_BASE_URL`, trimming whitespace and trailing slashes. - ReadwiseSettings gains an optional `baseUrl` field; it syncs as plaintext via the settings sync whitelist. - ReadwiseForm exposes the URL under a collapsed "Advanced" disclosure on the connect screen, and surfaces a custom URL read-only once connected. Disconnect preserves the custom URL for easy reconnect. Closes #4114 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * i18n(readwise): rename "Sync Base URL" label to "Custom URL" Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
5774e00c09 |
feat(sync): opt-in Credentials toggle + keyring v4 migration (#4111)
* feat(sync): add opt-in Credentials toggle to Manage Sync Adds a new Credentials category (default OFF) that gates the encrypted fields (OPDS / KOSync / Readwise / Hardcover usernames, passwords, and tokens) at both the publish and pull pipelines. When off, sensitive fields never leave the device, the proactive passphrase prompt never fires, and the Sync passphrase panel is hidden entirely. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * security: bump keyring to version 4 --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
1eae2af23e |
feat(sync): batch replica sync into one /api/sync/replicas request (#4109)
Auto-sync triggers (boot non-settings, focus, visibilitychange, online,
periodic) used to fan out N parallel `GET /api/sync/replicas?kind=…`
requests, one per replica kind. With 5 kinds today and the focus path
firing on every foreground transition, that's 5x the Cloudflare Worker
invocations of what the work actually requires.
Server: extend POST /api/sync/replicas to accept a batched-pull body
(`{ cursors: [{kind, since}, …] }`) alongside the existing push
(`{ rows: […] }`). Per-kind queries fan out via Promise.all — Supabase
calls inside the Worker aren't billed as Cloudflare requests, so DB
load is unchanged while Worker invocations collapse from N to 1.
Client/manager: add `client.pullBatch` and `manager.pullMany` that
share the existing cursor/HLC machinery. The boot path's `since=null`
override carries over via `pullMany(kinds, { since: null })`.
Orchestrator: `triggerIncrementalPullAll` now does ONE pullMany call
then fans out per-kind apply via Promise.allSettled. Boot does the
same for non-settings kinds (settings stays a single call to preserve
its apply-first ordering invariant).
Foreground triggers: listen to BOTH `focus` and `visibilitychange`,
sharing one throttle. focus is fastest on iOS Tauri WKWebView (~T=0,
~400ms ahead of visibilitychange). visibilitychange is the only
signal that fires on browser tab switching — focus does not. Drops
the Supabase user-ref-change listener (was the slowest of the three
foreground signals; redundant with the DOM events).
Bonus: `useBooksSync` now serializes `handleAutoSync` against
`pullLibrary` via the shared `isPullingRef` gate. The two paths used
to fire two concurrent `/api/sync?type=books` requests on the same
`since` value at startup; now whichever runs first claims the gate
and the other skips (throttle's `emitLast` retries afterwards).
Per session: boot 5→2 Worker calls. Per foreground trigger: 5→1.
|
||
|
Huang Xin
|
4110911011 |
fix(sync): keep dictionarySettings consistent across devices (#4105)
The bundled `settings` replica's `dictionarySettings.providerOrder`
and `providerEnabled` repeatedly drifted on multi-device setups: a
fresh-install Device B would overwrite Device A's authoritative
order with its own local default, dict tombstones referenced via
the settings replica left "skipped" gaps in the UI, and providerEnabled
keys missing from providerOrder rendered as silently lost imports.
Six related fixes (mostly orthogonal):
- **Disk-priming** in `initSettingsSync(initialSettings)`: seeds
`lastPublishedFields` from the just-loaded disk settings so the
first `setSettings(disk_default)` at boot diffs against the disk
baseline (no diff → no push), instead of diffing every whitelisted
field against `undefined` and clobbering the server with locals.
- **Settings boot pull is awaited first** in `useReplicaPull` (with
a shared `settingsBootPullPromise`) so the dict/font/texture/opds
pulls' auto-saves see server-primed `lastPublishedFields` rather
than disk defaults — implicit even when the caller didn't request
the `settings` kind.
- **Visibility / online / periodic auto-pull** in `useReplicaPull`:
module-level listeners with a 30s visibility throttle and a 5-min
interval keep long-lived foreground tabs in sync (previously the
hook only did the once-per-session boot pull and `ReplicaSyncManager.startAutoSync`'s
comments lied — it only flushed dirty pushes).
- **Tombstone scrubbing for no-local rows**: `softDeleteByContentId`
scrubs `providerOrder` / `providerEnabled` by contentId regardless
of whether a local dict matches, and `applyRow` always invokes it
on tombstones — so Device B fresh-installs that pulled tombstoned
contentIds via the settings replica without ever having a local row
still get the provider-side entries cleaned.
- **Orphan rescue** in `loadCustomDictionaries`: providerEnabled keys
that have no slot in providerOrder (per-field LWW splits a settings
push) get spliced before the first builtin so user-imported dicts
stay contiguous near the top of the list, not stranded after the
builtins where users miss them.
- **`addDictionary` prepends** to `providerOrder` so a fresh local
import shows up at the top of the list. Reviving a soft-deleted
entry preserves its existing slot.
- **Explicit-publish gate for `providerOrder`**: `markExplicitProviderOrderPublish()`
in `replicaSettingsSync` is the only way for `publishSettingsIfChanged`
to ship `dictionarySettings.providerOrder`. UI handlers that
intentionally reorder (drag-drop, dict import, dict delete,
web-search add) opt in via `saveCustomDictionaries(env, { publishOrderChange: true })`.
Auto-mutations from replica pull / orphan-rescue / tombstone-scrub
no longer ever republish the local view of order.
12 new tests across `replicaSettingsSync`, `replicaPullAndApply`,
`useReplicaPull`, and `customDictionaryStore`.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Huang Xin
|
302363a9fd |
feat(sync): per-category sync gates + Manage Sync UI (#4099)
* feat(sync): add per-category sync gates + Manage Sync UI The user can now enable / disable each sync category independently in Settings → Data Sync (User page). The map syncs across devices via the bundled `settings` replica, defaults to enabled so the preference is opt-out, and applies on both push (`replicaPublish`, legacy `useSync`) and pull (`useReplicaPull`, `useSync`) without backfilling on re-enable. Categories: - `book` / `progress` / `note` — gate the legacy `SyncClient` paths - `dictionary` / `font` / `texture` / `opds_catalog` — gate the replica-sync pulls + publishes for those kinds - `settings` — togglable, but force-on while `dictionary` is enabled because the dictionary's `providerOrder` / `providerEnabled` / `webSearches` live in the bundled settings replica. The UI shows the locked toggle as blue (enabled) with a hint instead of greying it out, since the underlying state IS on. UI: - New `SyncCategoriesSection` lists every category with a description and a daisyUI toggle. - New `Manage Sync` blue action on the User page (second slot, right after `Manage Subscription`); also surfaces inside the library `Advanced Settings` menu, deep-linking via `/user?section=sync`. - `SyncPassphraseSection` moved into the Manage Sync panel alongside the categories list. `Unlock now` button removed — the gate fires automatically on first encrypted push/pull and the manual unlock affordance was confusing. Adjacent cleanups: - `LangPanel` Dictionaries card gets `overflow-hidden` so the hover highlight clips to the card's rounded corners. - `FontPanel` gear icon replaced with a `Manage Fonts` row that matches the `Manage Dictionaries` pattern. i18n: extracted + translated 31 in-scope locales for the new strings (`Manage Sync`, `Data Sync`, `Manage Fonts`, plus the category copy block). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(i18n): translate new sync-categories strings across 30 locales Adds translations for the four strings extracted after the latest SyncCategoriesSection iteration: - `App settings` — toggle label for the bundled-settings sync gate - `Theme, highlight colours, integrations (KOSync, Readwise, Hardcover), and dictionary order` — description under that toggle - `Required while Dictionaries sync is enabled` — hint shown when the toggle is locked because dictionary sync depends on settings - `Unavailable` — `(Unavailable)` suffix on disabled translator providers; was missing from most locales until i18next-scanner picked it up this run Product names (KOSync, Readwise, Hardcover) left in Latin script. `Dictionaries` references in the third string reuse each locale's existing translation. `pt-BR` and `uz` deliberately untouched (out of the in-scope set). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(i18n): fill in pt-BR for the new sync-categories strings `pt-BR` is a registered, shipped locale (Portuguese (Brasil)) that fell through the gaps in earlier batch runs because it isn't listed in the i18n skill's locale-reference table. The fallback chain `pt-BR → pt → en` softened the impact, but BR-specific phrasing needs its own translations for the 20 new keys this PR added. `uz` stays excluded — that locale isn't registered anywhere (missing from i18next-scanner.config.cjs, src/i18n/i18n.ts, and TRANSLATED_LANGS), so its translation file is dead code. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(i18n): translate uz for the new sync-categories strings `uz` is a registered locale (listed in `i18n-langs.json` and `TRANSLATED_LANGS` as `'Oʻzbek'`) but earlier batch translation runs excluded it because the i18n skill's static locale-reference table was incomplete. Filling in the 20 strings this PR added. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(agent): update i18n skill --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
51a553dd89 |
feat(sync): bundle dictionary settings into the settings replica kind (#4096)
* feat(sync): bundle dictionary settings into the `settings` replica kind Adds three entries to the SETTINGS_WHITELIST so `providerOrder`, `providerEnabled`, and `webSearches` flow through the bundled settings replica with whole-field LWW. `defaultProviderId` (last- used tab) is deliberately excluded — it's per-device state. The customDictionaryStore exposes `applyRemoteDictionarySettings` so pulled values propagate into the in-memory mirror that the reader popup and the dictionary settings panel read from. Without this the mirror would stay stale until the next panel mount. Also fixes `saveCustomDictionaries`: it mutated the existing settings object in place and called `setSettings` with the same reference, so the replicaSettingsSync subscriber saw no change and never published. Build a fresh settings reference instead. Collapses the original PR 6 (`dict_provider_position`) and PR 7 (`dict_web_search`) plans, which proposed per-element CRDT rows with deterministic actor-id tiebreaks. Whole-field LWW is the right call given how rarely users edit these on two devices at once — same precedent as `customHighlightColors` and `customThemes` shipping through the bundled kind in PR 5. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(sync): make dict.id stable across devices (= contentId) Replaces the per-device `Math.random()` bundleDir as `dict.id` with the cross-device-stable `contentId`. `bundleDir` keeps tracking the device-local on-disk path, so on-disk layout is unchanged. Touch points: - 4 import paths in `dictionaryService.ts` + `buildLocalDictFromRow` in `replicaDictionaryApply.ts` set `id: contentId` instead of `id: bundleDir`. - Every `providerOrder` / `providerEnabled` entry that came from the dict store is now uniformly contentId-keyed, so the bundled `settings` replica syncs them across devices without any seam translation. Dicts with no contentId (very old, never synced) keep their bundleDir as id and remain local-only. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
6e7c9d1395 |
feat(sync): bundled settings replica kind for cross-device prefs and credentials (#4094)
* feat(sync): add bundled `settings` replica kind for cross-device prefs and credentials Adds a single-row `settings` replica that syncs a whitelist of `SystemSettings` fields across devices via per-field LWW (one entry per dot-namespaced path). Plaintext for theme / highlight colour / TTS configuration; encrypted (AES-GCM under the user's sync passphrase) for kosync / Readwise / Hardcover credentials. Highlights: - Push-side diff against an in-memory snapshot for plaintext paths and a localStorage SHA-256 hash for encrypted paths, so a refresh doesn't re-publish or re-prompt for the passphrase. - Pull-side cipher-fingerprint dedupe + per-row passphrase gate; decryption failures surface as toasts (wrong passphrase / orphan cipher) instead of silent drops. - Auto-recovery for orphaned ciphers: when a row references a saltId no longer in `replica_keys`, clear the local hash and re-encrypt under the current salt on the next save. - Single in-flight `/sync/replica-keys` fetch with a value cache to coalesce the boot-time burst of concurrent unlock callers. * fix(sync): guard settings dot-path helpers against prototype-polluting keys Reject `__proto__`, `constructor`, and `prototype` segments in the settings adapter's `readPath` / `writePath`. Every caller currently passes a constant from `SETTINGS_WHITELIST`, so the guard is purely defensive — but it silences the CodeQL prototype-pollution warning on PR #4094 and keeps the helpers safe if a future call site ever forwards an untrusted path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
712d564e9d |
feat(sync): encrypted OPDS credentials + Tauri keychain (PR 4c + 4d) (#4090)
* feat(sync): encrypt opds_catalog credentials end-to-end (TS path) Wires encrypted-credential sync for opds_catalog via the CryptoSession shipped in PR 4a (#4084) plus a new publish/pull crypto middleware. TS-only — native still uses ephemeral storage (re-enter passphrase per launch); PR 4d wires the OS keychain. - ReplicaAdapter gains optional `encryptedFields: readonly string[]`. Adapters stay sync; the middleware handles the crypto round trip. - replicaCryptoMiddleware.ts: encryptPackedFields drops the named fields from the push when the session is locked (no plaintext leak); decryptRowFields drops them on pull failure (local plaintext preserved by the store merge). - replicaPublish / replicaPullAndApply invoke the middleware. - OPDS adapter declares encryptedFields = [username, password] and now pack/unpack them as plaintext. - passphraseGate.ts: ensurePassphraseUnlocked coalesces concurrent calls, prompts via the registered prompter with kind=setup|unlock, throws NO_PASSPHRASE on cancel. - PassphrasePromptModal mounted at the Providers root; registers itself as the gate prompter. - CryptoSession.forget() wipes server-side envelopes + salts. - Migration 010 + replica_keys_forget RPC; DELETE /api/sync/replica-keys + client wrapper. - SyncPassphraseSection on the user page: status / Set / Unlock / Lock / Forgot. - CatalogManager pre-save: ensurePassphraseUnlocked when credentials are present; user cancel saves locally without sync. Plan updated: PR 4 split documented as 4a/4b/4c/4d. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(sync): persist sync passphrase via OS keychain (Tauri) Replaces the EphemeralPassphraseStore stub on native with real OS-keychain storage so users don't re-enter their sync passphrase every launch. Web stays on the in-memory ephemeral store by design. Native bridge plugin gains 4 commands wired across all platforms: - Rust desktop (`keyring` crate): macOS Keychain on apple-native, Windows Credential Manager on windows-native, Linux libsecret/ Secret Service on sync-secret-service. Per-target features so each platform compiles only the backend it needs. - iOS Swift: Security framework Keychain (kSecClassGenericPassword, SecItemAdd / Copy / Delete). - Android Kotlin: androidx.security EncryptedSharedPreferences (AndroidKeystore-derived AES-GCM master key, AES256_SIV / AES256_GCM key/value encryption). TS layer: - TauriPassphraseStore wraps the bridge calls. set is fail-loud (surfaces keychain rejection); get is fail-soft (returns null on any error so the gate prompts). - createPassphraseStore returns ephemeral synchronously; upgradeToKeychainIfAvailable swaps the singleton to TauriPassphraseStore on Tauri after probing the bridge. CryptoSession resolves the store via createPassphraseStore() each touch so the swap is transparent. - CryptoSession.tryRestoreFromStore: silent unlock at boot. Stale- entry recovery clears the store when the account has no salt server-side. unlock/setup persist; forget also clears the store. - Providers boot effect: upgrade keychain → silent restore. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(sync): make encrypted-credential pull actually decrypt + UX polish PR 4c shipped the encrypt path but the pull side silently dropped ciphers when locked, the modal was busy with double-rings, and web re-prompted on every page refresh. This rolls up the post-test fixes + UX polish: Pull-side decrypt: - decryptRowFields takes an `onLocked` callback the orchestrator wires to the passphrase gate; encountering a cipher field with a locked session now triggers the lazy-prompt path instead of dropping the field. - replicaPullAndApply re-applies the unpacked row for metadata-only kinds even when a local copy exists, so the now-decrypted creds reach the store (the binary-kind skip-if-local optimization doesn't apply). - Cipher fingerprint comparison: capture the row's `cipher.c` for each encrypted field, compare against the local record's lastSeenCipher. Same → skip prompt + decrypt entirely. Different (rotation / value change on another device) → prompt to re-decrypt. Fingerprint persists via OPDSCatalog.lastSeenCipher. Web persistence: - SessionStoragePassphraseStore: passphrase survives page refresh within the same tab, dies on tab close. Replaces EphemeralPassphraseStore as the default on web. Avoids localStorage / IndexedDB to keep the tab-scoped trust boundary. UI: - Renamed PassphrasePromptModal → PassphrasePrompt; modernized: filled input style with single subtle focus border, btn-primary + btn-ghost replaced with leaner custom buttons. eink-bordered + btn-primary classes give the dialog correct e-paper rendering. - globals.css: suppress redundant outline/box-shadow on focused text inputs / textareas (the element's own border is the focus indicator). - AGENTS.md: documents the e-ink convention (`eink-bordered`, `btn-primary` for inverted CTAs, etc.) so future widgets ship with e-paper support. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
6bfeb295d2 |
feat(sync): add opds_catalog replica kind (plaintext fields) (#4087)
Wires OPDS catalogs through replica sync as a metadata-only kind.
Plaintext fields only in this PR — encrypted credentials (username,
password) ship in the follow-up alongside the SyncPassphrasePanel UI
and Tauri keychain backend.
- Migration 009 extends the kind allowlist with 'opds_catalog'.
- replicaSchemas adds opdsCatalogFieldsSchema (name, url, description,
icon, customHeaders, autoDownload, disabled, addedAt) with a 50-row
per-user cap.
- New opdsCatalogAdapter is metadata-only (no `binary` capability).
Stable cross-device id from md5("opds:" + url.lower()) so two
devices that import the same URL converge to one row instead of
duplicating.
- New customOPDSStore (zustand) hydrates from SystemSettings,
publishes upserts/deletes through the replica pipeline, preserves
local-only username/password when overlaying remote updates, and
strips tombstones at the persistence boundary so existing
useSettingsStore readers (useOPDSSubscriptions, pseStream,
app/opds/page.tsx) need no migration.
- replicaPullAndApply branches on adapter.binary so metadata-only
kinds skip the bundleDir requirement and the manifest/binary path.
- CatalogManager rewires Add / Edit / Remove / Toggle / Add-popular
through the new store.
Plan update bundled in: tenet 8 (scalar settings sync via a bundled
row; collections sync per-record), per-kind allowlist now includes a
`settings` singleton that will collapse PRs 5 + 6+ into one bundled
adapter, and PR 4 is split into 4a (already merged) / 4b (this) / 4c
(encrypted credentials + UX).
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Huang Xin
|
4625e47a6d |
refactor(sync): extract shared adapter / pull-deps / legacy-migration primitives (#4081)
Three kinds (dictionary, font, texture) of replica sync now visibly duplicate each other; this PR extracts the shared shape now that the abstraction is well-validated, per the plan's "extract only after the second kind validates" guidance. - New `services/sync/adapters/_helpers.ts` — `unwrap` field-envelope, `singleFileFilenameFromManifest`, `singleFileBinaryEnumerator`, and a default `computeId` for kinds with `record.contentId`. Wired into font + texture adapters (dictionary stays bespoke — multi-file enumeration and a different identity recipe). - `useReplicaPull.ts` collapses the three near-identical `buildXPullDeps` (~80 lines each) into a single `buildReplicaPullDeps<T>` factory plus three small `ReplicaPullConfig` records (~12 lines each). Dispatch stays a typed `switch` to keep the generic record type sound under contravariance. - New `services/sync/migrateLegacy.ts` — `migrateLegacyReplicas<T>` helper that owns the rehash-flat-path → `<bundleDir>/<filename>` migration. `migrateLegacyFonts` and `migrateLegacyTextures` are now thin per-kind configs, ~15 lines each (down from ~60). Net: +143 / -291 across 5 files plus 2 new helpers. No behavior change; 3933 tests still pass. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
de6529523f |
feat(sync): cross-device background texture sync (#4079)
Plug the texture replica adapter into the kind-agnostic primitives shipped in #4077. Textures imported on one device download and become available on every signed-in device, with the same shape as the font sync stack (single-file binary, contentId from partialMD5+size+filename, bundleDir layout, replica-publish on import, full activation on auto-download). Includes legacy flat-path migration so pre-existing textures sync without re-import, ColorPanel import flow now publishes the row and queues the binary upload, and createCustomTexture preserves contentId/bundleDir/byteSize through addTexture (mirrors the font-import fix). Server allowlist gains 'texture' with a single-image Zod schema; useBackgroundTexture passes replica metadata through addTexture so the boot-time "ensure selected texture is in store" path doesn't silently un-publish a remote record. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
981579c255 |
feat(sync): cross-device custom font sync (#4077)
* refactor(sync): kind-agnostic replica primitives Extract dict-only sync into shared primitives (registry, pull/apply orchestrator, persist env, schema allowlist) so other kinds can plug in. Companion changes: per-replica Storage Manager grouping, useReplicaPull boot-race recovery, manifest=null reconciliation on every boot pull, copyFile takes explicit srcBase + dstBase, settled- event helpers, lenient webDownload Content-Length (R2/S3 signed URLs commonly omit it), and generic "File" transfer toast copy any replica kind can share. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(sync): cross-device custom font sync Plug the font replica adapter into the kind-agnostic primitives: font store gains replica wiring, custom font import publishes the replica row + queues a binary upload, and bootstrap registers the font adapter and download-complete handler. Includes legacy flat-path migration so pre-existing fonts sync without re-import, full @font-face activation on auto-download (load + mount the rule, mirroring manual import), and a fix to createCustomFont so contentId / bundleDir / byteSize survive the trip through addFont — otherwise import-time publish silently no-oped on missing contentId. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
cbdc3b8f52 |
feat(sync): wire dictionary store through replica sync (follow-up to #4075) (#4076)
* feat(sync): cross-device dictionary sync Custom MDict / StarDict / DICT / SLOB dictionaries now sync across signed-in devices via the replica layer. - Store mutations publish replica rows with field-level LWW + tombstones. - Re-importing the same content (renamed or after delete) preserves the user's label and reincarnates the server row instead of duplicating. - Manifest commits after binary upload so other devices never see a row whose binaries aren't on cloud storage yet. - Pull-side orchestrator creates a placeholder dict, queues the binaries via TransferManager, and clears the unavailable flag on completion. - Toast copy branches by transfer kind so dict uploads don't read "Book uploaded". Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(sync): boot pull and binary download path - Defer the boot pull until TransferManager is initialized so download enqueues aren't dropped. - Auto-persist the local dict store after applyRemoteDictionary; otherwise the next loadCustomDictionaries wipes the in-memory rows. - Boot pull passes since=null so a device whose cursor advanced past unpersisted rows can still recover. - Skip pulling when not authenticated instead of logging "SyncError: Not authenticated" on every boot of a signed-out device. - downloadReplicaFile resolves the destination against the kind's base dir; binaries previously landed at the literal lfp and openFile then failed with "File not found". Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(sync): per-page useReplicaPull hook Lifts the boot-time pull out of EnvContext into a hook each page mounts for the kinds it needs: useReplicaPull({ kinds: ['dictionary'] }). Library page and the shared Reader component opt in. The hook fires 10s after page load (so feature mounts hydrate first), dedups per-kind across navigation, and releases the slot on failure so a later mount can retry. Future kinds plug into the hook's per-kind switch. Also closes two refresh-loop bugs: - Hydrate the dict store from settings BEFORE the apply loop, so the auto-persist doesn't clobber persisted rows that the in-memory store hadn't yet read. Library-page refresh was the visible victim. - Skip the download queue when every manifest file is already on disk under the resolved bundle dir. Refreshing is a no-op; partial- download recovery still queues because some files would be missing. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
3b348c8f35 |
feat(sync): CRDT replica sync foundation (#4075)
* feat(sync): foundation for CRDT-based cross-device replica sync (Phase 1+2)
Adds the primitives and orchestration layer for syncing user-imported
assets (dictionaries, fonts, textures, OPDS catalogs, dict settings)
across devices via a polymorphic `replicas` table with field-level LWW
under HLC ordering. Phase 1 ships the foundation (CRDT, crypto, server
schemas, SQL migrations, push/pull endpoint); Phase 2 adds the adapter
registry, HTTP client, and sync manager. No modifications to existing
book sync — additive only.
Phase 1:
- src/libs/crdt.ts — HlcGenerator (monotonic + remote-absorption +
clock-regression-safe), per-field LWW with deviceId tiebreak,
remove-wins tombstones, reincarnation token revival.
- src/libs/crypto/{derive,encrypt,envelope,passphrase}.ts —
PBKDF2-600k key derivation (OWASP 2024), AES-GCM round-trip,
envelope {c,i,s,alg,h} with SHA-256 sidecar integrity check,
passphrase storage abstraction (web ephemeral; Tauri keychain stub).
- src/libs/replica-schemas.ts — Zod-backed allowlist (dictionary only
in PR 1), 64KiB row cap, 64-field cap, schemaVersion bounds,
filename validator.
- src/libs/replica-sync-server.ts — push batch validation
(auth + allowlist + schema + HLC ±60s skew clamp).
- src/pages/api/sync/replicas.ts — POST/GET endpoint wrapping the
Postgres crdt_merge_replica function via RPC.
- docker/volumes/db/migrations/003_add_replicas.sql — replicas table
+ replica_keys table + RLS.
- docker/volumes/db/migrations/004_crdt_merge_replica_fn.sql — atomic
per-field LWW merge function (forwards-compat preserves unknown
fields).
Phase 2:
- src/services/sync/replicaRegistry.ts — adapter contract
(core + optional BinaryCapability + LifecycleHooks per eng review).
- src/libs/replica-sync-client.ts — HTTP wrapper mapping status codes
to typed SyncError codes.
- src/services/sync/replicaSyncManager.ts — 5s debounced push,
immediate flush on visibilitychange/online, per-kind pull cursor,
remote HLC absorption.
Tests: 125 new (crdt 26, crypto 32, schemas 21, server 16, client 12,
registry 6, manager 12). Full suite 3656 passing, lint clean. Existing
book/config/note sync paths untouched.
Plan: ~/.claude/plans/vivid-orbiting-thimble.md
CEO plan: ~/.gstack/projects/readest-readest/ceo-plans/2026-05-06-replica-sync-cathedral.md
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(sync): add kind="replica" path through TransferManager (Phase 3)
Adds the replica branch to the existing book-shaped transfer
infrastructure so dictionary (and future kinds) bundles can flow through
the same queue, retry, and progress UI as book uploads.
Existing book transfer paths remain unchanged. The book-side regression
suite (37 tests in transfer-store.test.ts, 37 in transfer-manager.test.ts)
all stay green.
Store (src/store/transferStore.ts):
- TransferItem gains kind: 'book' | 'replica' (default 'book' on legacy
persisted rows), replicaKind, replicaId, replicaFiles, replicaBase.
- New addReplicaTransfer(replicaKind, replicaId, displayTitle, type, opts)
with files + base in opts; auto-computes totalBytes from file sizes.
- New getReplicaTransfer(replicaKind, replicaId, type) lookup.
- getTransferByBookHash filters to kind === 'book' (defensive against
bookHash="" collisions on replica items).
- restoreTransfers fills kind: 'book' for legacy persisted rows.
Manager (src/services/transferManager.ts):
- queueReplicaUpload / queueReplicaDownload / queueReplicaDelete.
- executeTransfer dispatches by kind to the new executeReplicaTransfer
(iterates files, calls appService.uploadReplicaFile per file with
per-file progress aggregation) or the existing executeBookTransfer
(refactored out, byte-identical behavior).
- Dispatches replica-transfer-complete event on success so stores can
react (e.g., commit manifest_jsonb to the replica row).
Storage / cloud (src/libs/storage.ts, src/services/cloudService.ts):
- uploadReplicaFile bypasses the book-only File.name smuggling and
takes an explicit cfp (cloud file path).
- uploadReplicaFileToCloud / downloadReplicaFileFromCloud /
deleteReplicaBundleFromCloud orchestrate per-file operations under
${userId}/Readest/replicas/<kind>/<replicaId>/<filename>.
- replicaCloudKey() centralizes the path-construction rule.
- New CLOUD_REPLICAS_SUBDIR constant.
App service (src/services/appService.ts, src/types/system.ts):
- AppService gains uploadReplicaFile, downloadReplicaFile,
deleteReplicaBundle (file-level operations; orchestration lives in
TransferManager).
Tests: 18 new (12 in transfer-store.test.ts, 5 in transfer-manager.test.ts,
1 fixture). Full suite 3674 passing, lint clean. Existing book regression
clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(sync): dictionary replica adapter + bootstrap (Phase 4, partial)
Lands the safe-to-ship foundation of Phase 4 — adapter logic, registry
bootstrap, and SystemSettings hooks for replica sync. The on-disk
migration of legacy customDictionaries (bundleDir → content-hash id),
the live store wiring, and the Settings → Sync UI are deferred to a
follow-up PR so they can land with real-device QA.
Adapter (src/services/sync/adapters/dictionary.ts):
- dictionaryAdapter: kind='dictionary', schemaVersion=1.
- pack/unpack — only synced subset (name, kind, lang, addedAt,
unsupported{,Reason}). bundleDir / files / unavailable / deletedAt
stay per-device or are handled by the tombstone mechanism.
- BinaryCapability.enumerateFiles dispatches by bundle kind:
- mdict: mdx + mdd[] + css[]
- stardict: ifo + idx + dict + syn (skips .idx.offsets / .syn.offsets
sidecars — those are device-local indices)
- dict: dict + index
- slob: single .slob file
- primaryDictionaryFile() picks the anchor file per kind for
partialMD5 hashing.
- computeDictionaryReplicaId(partialMd5, byteSize, sortedFilenames)
produces a deterministic 32-hex content-hash id used at import time.
- 23 tests cover pack/unpack identity, kind dispatch, file enumeration,
id determinism, and per-device-field exclusion.
Bootstrap (src/services/sync/replicaBootstrap.ts):
- bootstrapReplicaAdapters() registers all known adapters once at app
start. Idempotent (safe to call multiple times). Wired into
EnvContext.tsx so the registry populates on app mount.
- 3 tests cover registration, idempotency, and the PR-1 allowlist.
SystemSettings (src/types/settings.ts, src/services/constants.ts):
- +SyncCategory = 'book' | 'progress' | 'note' | 'dictionary' — typed
union for the user-facing sync toggles. 'progress' gates the
existing book-config sync (reading progress); 'note' gates
annotations; 'book' gates book binaries + metadata; 'dictionary'
gates the new replica sync. Future replica kinds extend the union.
- +SYNC_CATEGORIES readonly array for UI iteration.
- +syncCategories: Partial<Record<SyncCategory, boolean>> — per-
category opt-in toggles in DEFAULT_SYSTEM_SETTINGS (default ON for
all four). UI panel ships in the follow-up.
- +lastSyncedAtReplicas: Record<string, string> — per-kind HLC pull
cursors (matches replicaSyncManager's CursorStore contract).
Registry type cleanup (src/services/sync/replicaRegistry.ts):
- BinaryCapability.enumerateFiles return shape: localRelPath → lfp
to match the existing TransferStore.ReplicaTransferFile convention.
Tests: 28 new (23 dict + 3 bootstrap + 2 syncCategories defaults).
Full suite 3702 passing, lint clean. Existing book/config/note sync
paths untouched.
Deferred to PR 1 follow-up (with real-device QA):
- customDictionaryStore migration: rehash legacy uniqueId() bundleDir
to content-hash id; preserve providerOrder mapping; staged
.legacy/<old-id>/ backup.
- Wire customDictionaryStore mutations through replicaSyncManager
(markDirty on add/rename/delete; pull on init).
- Settings → Sync panel: per-category toggles + last-sync timestamps.
- Sync passphrase modal: set / change / forgot flow (lazy first prompt
on encrypted-field push/pull).
- <CloudReplicaRow> in CustomDictionaries.tsx for "Download from cloud
(X MB)" affordance.
- Tauri keychain backend for sync passphrase storage.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(sync): replicaSync singleton + content-hash id at dict import (Phase 4b)
Two foundation pieces that everything UI-side will sit on top of, both
purely additive — no live store wiring, no behavior change for existing
dictionary imports.
replicaSync singleton (src/services/sync/replicaSync.ts):
- initReplicaSync({deviceId, cursorStore, hlcStore?, client?}) builds
one ReplicaSyncManager backed by an HlcGenerator with persistence
wrapped around .next()/.observe(). Idempotent (second init returns
the existing instance).
- LocalStorageHlcStore (src/libs/hlc-store.ts) snapshots the HLC
counter under 'readest_replica_hlc' so it survives restart. Falls
back silently when localStorage is unavailable (private mode, SSR);
client re-derives via the existing remote max(updated_at_ts) repair
path. InMemoryHlcStore is the test backend.
- 16 tests (9 hlc-store, 7 replicaSync) covering snapshot persistence,
restore-on-init, and idempotency.
- Wiring into EnvContext for production deferred to the follow-up that
also adds the cursor store backed by useSettingsStore.
Content-hash id at dictionary import
(src/services/dictionaries/contentId.ts):
- computeDictionaryContentId(primaryFile, filenames) wraps
computeDictionaryReplicaId(partialMd5(primary), byteSize,
sortedFilenames) — the cross-device id used as the replica_id when
the dict actually pushes/pulls.
- Wired into all four import paths in dictionaryService.ts:
- stardict primary = .ifo (small text, partialMD5 ≈ full hash)
- mdict primary = .mdx (body)
- dict primary = .dict.dz (gzipped body)
- slob primary = .slob (single-file bundle)
ImportedDictionary gains contentId?: string. Optional for backwards
compat; legacy bundles without contentId are flagged as
"needs rehash before sync" by the upcoming store-wiring follow-up.
- 6 tests cover identity determinism, byteSize sensitivity, filename-
set sensitivity, and order-independence.
Tests: 22 new (9 + 7 + 6). Full suite 3724 passing, lint clean.
Existing dictionary import flow unchanged for users — contentId is an
additional field, not a replacement for the bundleDir-based id.
Deferred to follow-up (with on-device QA):
- Production cursor store backed by useSettingsStore +
appService.saveSettings.
- EnvContext call to initReplicaSync after appService boot.
- customDictionaryStore mutation hooks → replicaSyncManager.markDirty.
- Legacy bundleDir → contentId migration with .legacy/ backup.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(sync): land replica-sync design plan in repo
Moves the plan document that drove this PR's foundation work
(`~/.claude/plans/vivid-orbiting-thimble.md`) into the project tree at
`apps/readest-app/.claude/plans/` so reviewers and future contributors
can read it alongside the code without leaving the repo.
The plan went through three review passes — Codex (19 findings, all
absorbed), CEO/scope review (mode SCOPE EXPANSION; encrypted secrets
pulled forward to v1, "private-only forever" posture lock), and eng
review (FULL_REVIEW mode, 16 findings absorbed). The full review trail
lives in the file's `## GSTACK REVIEW REPORT` section at the bottom.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(crdt): point README at in-repo plan path
Now that vivid-orbiting-thimble.md lives at
apps/readest-app/.claude/plans/, the README link should point there
rather than at the home-dir copy that no longer exists.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(sync): rename src/libs files to camelCase per project convention
Test files renamed in lockstep. Imports + comment references updated
across cloudService, storage, transferManager, replicaSync,
replicaSyncManager, /api/sync/replicas, and all four test files. No
behavior change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Zach Bean
|
e8f6db96e4 | fix(kosync): CWA sometimes sends 400 for auth failure (#3893) | ||
|
Zach Bean
|
50a2957e35 |
feat(kosync): support HTTP Basic auth for CWA KOSync servers (#3792)
* Support HTTP Basic auth for kosync connections * fix(kosync): use separate password field for HTTP Basic auth - Add `password?` field to KOSyncSettings to store the plain password alongside the existing `userkey` (md5 hash), preserving backward compatibility for existing users - Replace Buffer.from() with btoa() for browser-compatible Base64 encoding - Simplify buildHeaders() to use config fields directly instead of the useAuth union type Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Huang Xin <chrox.huang@gmail.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
e0e991b599 | fix(kosync): proxy kosync request only for the web app, closes #2440 (#2466) | ||
|
Huang Xin
|
6003eeadba | compat: support more kosync server implementations, closes #1862 (#1929) | ||
|
Huang Xin
|
584f3511f8 | refactor(kosync): move kosync into reader sidebar menu and add manual pull/push trigger (#1912) | ||
|
Huang Xin
|
d390209dab | feat: support importing TTF/ODF fonts, closes #979 and closes #1708 (#1864) | ||
|
Huang Xin
|
079aeaedb0 | fix(sync): support custom koreader sync server hosted in LAN, closes #1800 (#1830) | ||
|
Zeedif
|
595608bd62 |
feat: Implement KOReader Progress Synchronization (#1770)
* feat(sync): implement KOReader progress synchronization This commit introduces a comprehensive feature to synchronize reading progress with a KOReader sync server. It includes a compatibility layer to handle discrepancies between Readest's CFI-based progress and KOReader's XPointer/page-based progress, primarily using the `percentage` field as a common ground. Key additions include: - A new settings panel under "KOReader Sync" for server configuration, authentication, and sync strategy management (e.g., prompt on conflict, always use latest). - A conflict resolution dialog that appears when remote progress differs significantly from local progress, allowing the user to choose which version to keep. - A client-side `useKOSync` hook to manage the entire synchronization lifecycle, including API calls, state management, and conflict resolution logic. - A new API endpoint `/api/kosync` that acts as a secure proxy to the user-configured KOReader sync server, handling authentication and forwarding requests. - Logic to differentiate between paginated (PDF/CBZ) and reflowable (EPUB) formats, using page numbers for paginated files where possible and falling back to percentage for reliability. - Spanish translations for all UI elements related to the KOReader sync feature. - Addition of `uuid` package to generate a unique `device_id` for sync purposes. Refactor: - The `debounce` utility has been improved to include `flush` and `cancel` methods, allowing for more precise control over debounced function execution, which is now used in the sync hook. * fix(kosync): add support for converting between XPointer and CFI in progress synchronization * fix(kosync): update navigation method to use select instead of goTo for paginated formats * fix(kosync): refactor synchronization settings and improve conflict resolution handling * fix(kosync): add event dispatcher for flushing KOReader synchronization * fix(sync): handle xpointer in a different section, fix styling * i18n: update translations --------- Co-authored-by: Huang Xin <chrox.huang@gmail.com> |