ccb937015d841cfa05fbb66aa1901c9aee005d9d
6 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Huang Xin
|
ccb937015d |
feat(sync): incremental file sync and per-book transfers for the active provider (#4982)
* fix(sync): record remote-present files for no-source books in the upload cursor With "Upload Book Files" on, a device that holds no local copy of a book (e.g. the web app with a cloud-only library) HEAD-probed the remote for every book on every sync: pushBookFile returned 'no-source' and the hash was never recorded in library.json's uploadedHashes, so needsFilePush stayed true for the whole library and each run (tab focus, Sync Now, library change) issued one Drive/WebDAV request per book, 646 requests per sweep in the reported case. The HEAD probe already answers whether the file is on the remote. Carry that in PushBookFileResult.remoteExists and record the hash when a no-source book's file is already mirrored, so the next incremental sync skips it and stays O(changed). Books absent both locally and remotely stay unrecorded so a device that has the bytes can upload them later. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(sync): reach the no-source verdict without probing the remote A book file sync with "Upload Book Files" on probed the remote for every book before checking whether this device even holds the bytes. On a device with a cloud-only library (the web app), none of the books have a local source, so every sync run (tab focus, Sync Now, library change) issued one name-lookup request per book against Google Drive, a full per-book request storm that never converged: with no local file there is nothing to upload and nothing to record, so the next run repeated it. Resolve the local source first and return 'no-source' from local state alone; the remote head probe now runs only when there is a local file to compare or upload. The probe keeps its non-NETWORK rethrow semantics so the auth-failure latch (#4981) still stops a run on an expired session. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(sync): incremental file sync and per-book transfers for the active provider Cut the redundant remote work a file-sync run does and route explicit per-book uploads/downloads to the active third-party provider (WebDAV / Google Drive) instead of the gated Readest Cloud transfer queue. Engine (services/sync/file/engine.ts, wire.ts): - etag change-probe: one HEAD on library.json, cached per provider for the session. When the etag matches the last successful pull, reuse the cached index and skip both the index download and the discovery scan. An AUTH failure on the probe aborts the run like the full pull does. - emptyDirs memo carried in the index: dirs found to hold no book file are recorded so clients stop re-listing them every run. Re-checked when the file arrives (uploadedHashes), on Full Sync, or when a legacy client drops the record; pruned only against a listing that actually ran. - skip the index re-push when the rebuilt index is semantically identical to the pulled one, so a restamped byte-copy no longer churns the remote and invalidates peers' etag change detection. - downloadBookFile() for the explicit per-book Download action. Provider reuse (services/sync/file/providerRegistry.ts): - memoise one provider per connection key, shared by every surface (reader per-book sync, library auto-sync, Sync now / pull to refresh). Reuses the Drive path->id cache instead of re-resolving /Readest, books/ and library.json by name query on every engine build. Drive connect/disconnect resets the cache since its token source changes identity with no key input changing. Google Drive (services/sync/providers/gdrive/GoogleDriveProvider.ts): - write fast-path: PATCH a path whose id is cached in place with no lookup, falling back to a full resolve on a 404 (stale id). Removes a files.list per PUT in the steady state. - dev-only request diagnostics: one line per provider op and per HTTP attempt so a run's request budget can be attributed from the console. Per-book transfers (services/sync/file/runLibrarySync.ts): - runActiveFileBookUpload / runActiveFileBookDownload build the active provider's engine and push/pull a single book, stamping downloadedAt like the native path. Wired into the reader/library book actions with toasts. UI, status, and i18n: - Readest Cloud sub-page: drop the quota stats and wrap the "Account and Storage" row in the BoxedList primitive so it aligns to the design system. - Shorten provider status/toast copy ("Active", "Google Drive session expired", "Library sync via {{provider}}", "KOReader") and translate the new keys across all locales. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Huang Xin
|
57868a138e |
feat(settings): unified Cloud Sync chooser with Readest Cloud as a first-class provider (#4976)
The Integrations section is now one Cloud Sync chooser: Readest Cloud,
WebDAV, and Google Drive as radio rows, Readest Cloud first, with a
scope subtitle stating what the choice governs (library data, on this
device) and what always stays with the Readest account (settings,
statistics, dictionaries).
- Activation helpers move to services (cloudSyncActivation.ts) and
accept 'readest' (= no third-party provider active); the component
module re-exports for existing imports.
- Row status lines come from a pure, fully-tested state matrix
(cloudSyncStatus.ts) so every string is enumerated in one place:
signed-out / loading / active / available for Readest Cloud;
not connected / configured / paused / syncing / sync failed /
book-file-uploads-off warning / active for third-party rows. The
paused state renders on the affected third-party row (the plan sketch
placed it on the Readest row; the provider that is paused is the
third-party one).
- The Readest Cloud row opens an inline sub-page (SubPageHeader + the
storage/translation Quota + a NavigationRow out to Account) instead of
ejecting from the Settings dialog; signed-out taps go to login and the
radio is unchecked and disabled, so an idle signed-out state never
shows a checked radio while nothing syncs.
- Premium-gated builds keep the Readest Cloud row and show the upgrade
prompt only in place of the third-party rows.
- Two-direction capability Tips in the WebDAV/Drive sub-pages spell out
both what syncs only to the user's server and what still flows through
the Readest account; the Upload Book Files description notes that
Readest Cloud uploads pause while the provider is selected.
- Manage Sync book/progress/note rows swap their description to
'Managed by {{provider}}...' via the existing locked-row pattern while
a third-party provider is selected; the toggles stay interactive and
persist since they govern the native channels after switch-back.
- The chooser rows form a radiogroup (native same-name radios provide
arrow-key group movement) with an accessible group label.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Huang Xin
|
ea99106677 |
fix(sync): silence third-party cloud-sync error toasts (#4845)
* fix(sync): never toast third-party cloud-sync errors; log to console only The reader's per-book auto-sync surfaced an "Cloud sync authentication failed. Reconnect in Settings." toast on any AUTH_FAILED (e.g. an expired web Google Drive token), interrupting reading. Background sync failures shouldn't pop a toast — drop it and console.warn every sync error instead (the AUTH_FAILED branch only chose toast-vs-console, so it collapses to a plain log). Removes the now-unused authFailedToast + useTranslation/FileSyncError imports. Manual "Sync now" (FileSyncForm) still reports its result — it's a deliberate, foreground action. Native cloud sync (useBooksSync) is unaffected. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(sync): surface an expired cloud-sync session in the reader + Settings With sync-error toasts silenced, an expired third-party session (e.g. the short-lived web Google Drive token) had no UI indicator. Surface it without the old per-failure error toast: - Reader: a single top-right `hint` ("Google Drive session expired. Reconnect in Settings.") — the same affordance as the native "Reading Progress Synced" hint. De-duplicated via a per-instance ref so it shows once, not on every page-turn sync; reset on a successful sync / provider switch (web reconnect reloads anyway). - Settings → Google Drive: Disconnect swaps to Reconnect when the session is expired, and "Sync now" is disabled (FileSyncForm gains a `syncNowDisabled` prop) so a sync that would just fail isn't offered. No hint text in Settings. - webTokenStore.hasValidWebDriveToken() backs the web detection (the token lives in sessionStorage; native auto-refreshes so it doesn't apply there). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
5f44c95592 |
feat(sync): library-scoped auto-sync for third-party cloud (WebDAV / Drive) (#4835)
Parity with native useBooksSync: keep library.json current on import, delete, and book-close, not just on a manual "Sync now". - useLibraryFileSync: new library-scoped hook (counterpart of useBooksSync), mounted once on the library page. Builds the active provider's engine async and runs engine.syncLibrary on every library change (import adds a row, delete sets deletedAt, closing a book bumps updatedAt), debounced 5s and gated on the global file-sync mutex + Sync Strategy + Upload Book Files. The reader's per-book useFileSync is unchanged (it's the per-book progress sync). - Pass the FULL library (incl. soft-deleted books) to engine.syncLibrary, in both the new hook and the manual FileSyncForm "Sync now": the engine tombstones deleted books in library.json so deletions propagate, and keeping them in the input set stops the discovery pass from re-downloading a book the user just deleted (its remote hash dir lingers until the GC sweep). - Tests: engine tombstones a soft-deleted book in the pushed index and does not re-download one whose remote dir still exists. Gated only by the active provider's enabled flag + strategy (cloud sync is currently ungated from premium). Never runs before the library loads from disk, so it can't push an empty index over the remote. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
d932444b78 |
fix(sync): cloud-sync settings polish + temporary premium ungate (#4828)
* fix(settings): clamp option-row description to a single line SettingsRow descriptions wrapped to multiple lines on narrow (mobile) widths, giving boxed-list rows uneven heights (e.g. "Uploads book files to your other devices." in the Cloud Sync panel). Clamp the description to one line with ellipsis in the shared primitive so every option row stays uniform; the description is a hint, not a paragraph (longer copy belongs in a Tips block). Codified in DESIGN.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * i18n(settings): shorten sync strategy labels to "Send only" / "Receive only" Rename the Sync Strategy options (shared by the Cloud Sync and KOReader Sync forms). Keys renamed in every locale, preserving existing translations. * feat(sync): temporarily ungate third-party cloud sync from premium Cloud sync (WebDAV / Google Drive) ships available to every plan, incl. free, while the feature stabilises. Gated behind a single CLOUD_SYNC_REQUIRES_PREMIUM flag (off) via isCloudSyncAllowed; the paywall code (CLOUD_SYNC_PLANS / isCloudSyncInPlan) is intact, so re-gating in an upcoming release is a one-line flip. Applies to the Settings provider rows and the reader auto-sync gate. * fix(settings): polish cloud-sync connect buttons Use btn-contrast for the WebDAV and Google Drive Connect CTAs (theme-neutral, e-ink correct); rename "Connect Google Drive" to "Connect"; move the Google Drive sign-in tips below the Connect button. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
7e78f80e14 |
feat(sync): Google Drive cloud sync + premium Third-party Cloud Sync section (desktop) (#4821)
* feat(sync): add Google Drive file-sync provider core Second FileSyncProvider for the merged provider-agnostic file-sync engine, behind the provider seam. This is the CI-testable core only: no settings UI and no platform OAuth runners yet (those land in later phases). - GoogleDriveProvider over the Drive v3 REST API: id-addressed path resolution with a per-instance id cache, create-then-name uploads, real idempotent ensureDir, files.list pagination, Retry-After-aware 429/5xx backoff, per-path folder-creation locks with deterministic duplicate collapse, stale-id eviction, and FileSyncError mapping (403 split into rate-limit vs permission). - DI OAuth layer: pkce, parseRedirect (redirect-target + CSRF state), reverseDnsRedirect, tokenStore (iOS client, no secret), oauthFlow. - PersistedDriveAuth with single-flight token refresh; keychain-backed token store with no ephemeral fallback for the refresh token; account label via about.get. - providerRegistry (backend kind to provider) and buildGoogleDriveProvider assembly. - Shared transport-agnostic provider semantic contract, run against both WebDAV and Drive. - Keyed secure-KV bridge contract (set/get/clear_secure_item); the native keychain implementation lands with the desktop OAuth slice that first exercises it. Adapted from ratatabananana-bit/Readest-google-drive-mod-patcher (AGPL-3.0) with the author's explicit permission. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(sync): multi-provider file-sync settings + sync-state foundation PR2 foundation for a second file-sync backend (Google Drive). The behaviour-sensitive reader-hook and Sync-now form generalization land in PR3 alongside OAuth, where Drive actually connects and the multi-provider paths can be exercised and live-verified (and the extracted form gets its second consumer, avoiding a single-use abstraction). - GoogleDriveSettings type (mirrors WebDAVSettings minus URL/credentials/ rootPath, plus accountLabel) wired into SystemSettings, with DEFAULT_GOOGLE_DRIVE_SETTINGS in the defaults. - googleDrive.deviceId + googleDrive.lastSyncedAt added to the backup blacklist so device-local sync identity / cursors never restore onto another device. Covered by the existing backup-settings test. - Generalize webdavSyncStore into fileSyncStore: per-backend progress keyed by provider kind, plus a global library-sync mutex (beginSync returns false when another backend already holds the lock) since every backend's syncLibrary mutates the same local library. Migrate WebDAVForm and IntegrationsPanel to the keyed API; WebDAV behaviour is unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(native-bridge): add keyed secure key-value store commands A generic, keyed secret store over the same OS keychain backends as the sync passphrase (set/get/clear_secure_item), so secrets that aren't the single sync passphrase get the same XSS-free cross-launch persistence without each needing its own native command. The Google Drive OAuth token store (PR1's KeychainTokenPersistence) is the first consumer; a future cloud provider's refresh token reuses it. - Desktop (macOS/Windows/Linux): keyring-core, keyed by the item key as the entry account under the existing "Readest Safe Storage" service. - Android: EncryptedSharedPreferences (a dedicated readest_secure_items_v1 file, the item key as the pref key). - iOS: Security framework Keychain (kSecClassGenericPassword, dedicated service, the item key as kSecAttrAccount). Registered in the plugin invoke handler + build COMMANDS + default permission set (autogenerated permission files regenerated; the passphrase entries are preserved). The TS bridge wrappers shipped in PR1. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(sync): desktop Google Drive OAuth runner + connect flow The desktop half of Drive sign-in: open consent in the system browser, capture the reverse-DNS redirect the OS routes back, and exchange the code for tokens. - oauthDesktop.ts: runDesktopDeepLinkOAuth wires the DI OAuth flow to the desktop mechanics (open default browser, capture via single-instance / onOpenUrl, cold-browser fallback after a grace period, hard deadline). Fully headless-unit-tested via injected deps. - spawn_fresh_browser.rs (+ registration, Windows-only winreg dep): the cold browser the runner falls back to when the user's already-running browser snapshotted protocol associations before the scheme was registered (a Windows-specific failure). Resolves the default browser from the registry and spawns it cold with an isolated --user-data-dir; a no-op on macOS/Linux where the default-browser open already routes the redirect. Pure helpers unit-tested. - connectGoogleDrive.ts: run the platform OAuth runner, persist the token (fail-loud — Drive is not reported connected if the refresh token does not save), and resolve the account label via about.get (best-effort). OAuth runner adapted from ratatabananana-bit/Readest-google-drive-mod-patcher (AGPL-3.0) with the author's permission. Scheme registration + the ingress redirect filter + the Drive connect UI land in the following commits; live desktop verification follows once the official Google client id is provisioned. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(sync): filter Google OAuth redirects out of the deep-link ingress The reverse-DNS OAuth redirect (com.googleusercontent.apps.<id>:/oauthredirect) is delivered through the same single-instance / onOpenUrl channels as book-file deep links. Without a filter the book-import consumer would treat the redirect URL as a file path to open. Drop it at the ingress source (useAppUrlIngress) before the app-incoming-url broadcast, so no consumer ever sees it; the Drive sign-in runner still captures it via its own listeners. isGoogleOAuthRedirectUrl matches the scheme prefix (not a specific client id), so it stays correct regardless of which client is baked into the build. Note: registering the scheme in tauri.conf.json (so the OS routes it back to the app) needs the official Google client id, which is a provisioning prerequisite. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(sync): bake the official Google Drive OAuth client id + redirect scheme Provisioned the Readest Google Cloud OAuth client (iOS application type, no secret, drive.file scope). Bake the client id as the default in getGoogleClientId (overridable via NEXT_PUBLIC_GOOGLE_CLIENT_ID for forkers, who must also regenerate the manifest schemes) and register the derived reverse-DNS redirect scheme com.googleusercontent.apps.<id> in tauri.conf.json (desktop + mobile deep-link) so the OS routes the OAuth redirect back to the app. The client id is a public client identifier, not a secret. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(sync): Google Drive connect UI + shared FileSyncForm Make Drive usable from Settings, and extract the now-two-consumer sync controls. - FileSyncForm: the provider-agnostic sync controls (sub-toggles, conflict strategy, manual "Sync now" with progress + result toast), parameterised by backend kind and building the provider through the registry. Extracted from WebDAVForm now that a second consumer exists. WebDAVForm keeps its URL/credentials connect panel + browse pane and renders FileSyncForm for the sync section; behaviour is unchanged (WebDAV "Sync now" goes through the same provider via the registry). - GoogleDriveForm: an OAuth connect panel (Connect -> runGoogleDriveConnect -> store token in keychain -> "Connected as <email>"; Disconnect) + FileSyncForm. - googleDriveConnect.ts: assemble the env client id + keychain + desktop runner into connectGoogleDrive/disconnectGoogleDrive for the UI. - IntegrationsPanel: a "Google Drive" row + sub-page, shown only on desktop (mobile OAuth runners land in later phases). Reader-side auto-sync (generalizing useWebDAVSync) is a follow-up; manual "Sync now" already exercises the full Drive stack. Full suite 6412 green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(settings): unified Third-party Cloud Sync section (exclusive provider) Group WebDAV + Google Drive into a new "Third-party Cloud Sync" section and make them mutually exclusive — only one cloud provider syncs the library at a time. - New unified "Cloud Sync" sub-page (CloudSyncForm): a provider picker (radio, the AIPanel mutually-exclusive pattern) on top, the shared FileSyncForm sync options below for whichever provider is active. Google Drive is offered only on desktop; on mobile the page is WebDAV only and the picker is hidden. - withActiveCloudProvider helper: enabling one provider disables the other in one save. Both panels' connect/activate paths use it. Unit-tested. - WebDAVForm / GoogleDriveForm refactored into embeddable panels (the unified page owns the header). Drive gains a "configured but inactive" state so switching back re-activates it without a fresh sign-in; explicit Disconnect clears the keychain token. - IntegrationsPanel: remove the two separate WebDAV / Google Drive rows from "Reading Sync" (now KOReader Sync / Readwise / Hardcover only); add the Third-party Cloud Sync section with one Cloud Sync row (status = active provider). Old webdav/gdrive deep-links route to the unified page. Also removes the temporary Drive concurrency probe (the upload already runs at the intended concurrency 4; the probe confirmed it). Full suite 6416 green; lint + format clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(reader): auto-sync the active cloud provider while reading Generalize the reader sync hook from useWebDAVSync to useFileSync so the active third-party cloud provider (WebDAV OR Google Drive) syncs per-book while reading — pull-on-open, debounced push on progress/booknote changes, cover/file upload — not just via the manual "Sync now" in settings. Since the providers are mutually exclusive, the hook drives exactly the one enabled backend, built through the provider registry. The build is async (the Google Drive provider probes the OS keychain), so the engine lives in state and the pull-on-open waits for it; switching providers mid-session resets the per-book locks. The engine is keyed on connection-relevant settings so a lastSyncedAt write doesn't re-probe the keychain. deviceId / lastSyncedAt now write the active provider's settings slice; the auth-failed toast is provider-neutral; the per-book events are renamed *-file-sync. WebDAV reader-sync behaviour is unchanged. Full suite 6416 green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(settings): surface cloud providers in the section with inline switch Show WebDAV + Google Drive as separate rows in the Third-party Cloud Sync section (instead of one "Cloud Sync" row), so both providers are visible and the active one can be switched right there. - CloudProviderRow: a trailing radio makes a provider the single active sync target inline (enabled only when it's already configured — WebDAV creds / a Drive token); the row body / chevron opens its config sub-page (connect, sync options, disconnect). Status reads Active / Configured / Not connected, with a Syncing… indicator. - Each provider drills into its own sub-page again (WebDAV / Google Drive), rendering the embeddable panel under a SubPageHeader; the brief unified CloudSyncForm picker page is removed (its old deep-link maps to Google Drive). - Switching stays exclusive via withActiveCloudProvider; an inline switch trusts the stored credentials/token (no re-validate / re-OAuth). Full suite 6416 green; lint + format clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(sync): gate third-party cloud sync behind a premium plan WebDAV + Google Drive sync is now a premium feature: available on any paid plan (Plus, Pro, or Lifetime), not on free. - isCloudSyncInPlan(plan) helper (mirrors isEmailInPlan; plus/pro/purchase). - IntegrationsPanel: free users see the Third-party Cloud Sync section with an upgrade row ("Available on Plus, Pro, or Lifetime") that opens the plans page instead of the provider rows; the cloud-sync deep-links are gated too (waiting for the plan to load before deciding). - useFileSync: the reader's auto-sync only runs on a paid plan, so a downgraded user's sync stops even if a provider's enabled flag lingers. Full suite 6418 green; lint + format clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(sync): escape backslashes in Drive query literals (CodeQL) escapeDriveLiteral escaped single quotes but not the backslash escape character, so a file name containing a backslash (or ending in one) could break out of the single-quoted Drive `files.list` query literal and malform the query. Escape backslashes first, then single quotes, so the backslashes added for the quotes are not doubled. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |