Files
readest/apps/readest-app
Huang Xin ccb937015d feat(sync): incremental file sync and per-book transfers for the active provider (#4982)
* fix(sync): record remote-present files for no-source books in the upload cursor

With "Upload Book Files" on, a device that holds no local copy of a book
(e.g. the web app with a cloud-only library) HEAD-probed the remote for
every book on every sync: pushBookFile returned 'no-source' and the hash
was never recorded in library.json's uploadedHashes, so needsFilePush
stayed true for the whole library and each run (tab focus, Sync Now,
library change) issued one Drive/WebDAV request per book, 646 requests
per sweep in the reported case.

The HEAD probe already answers whether the file is on the remote. Carry
that in PushBookFileResult.remoteExists and record the hash when a
no-source book's file is already mirrored, so the next incremental sync
skips it and stays O(changed). Books absent both locally and remotely
stay unrecorded so a device that has the bytes can upload them later.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(sync): reach the no-source verdict without probing the remote

A book file sync with "Upload Book Files" on probed the remote for every
book before checking whether this device even holds the bytes. On a
device with a cloud-only library (the web app), none of the books have a
local source, so every sync run (tab focus, Sync Now, library change)
issued one name-lookup request per book against Google Drive, a full
per-book request storm that never converged: with no local file there is
nothing to upload and nothing to record, so the next run repeated it.

Resolve the local source first and return 'no-source' from local state
alone; the remote head probe now runs only when there is a local file to
compare or upload. The probe keeps its non-NETWORK rethrow semantics so
the auth-failure latch (#4981) still stops a run on an expired session.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(sync): incremental file sync and per-book transfers for the active provider

Cut the redundant remote work a file-sync run does and route explicit
per-book uploads/downloads to the active third-party provider (WebDAV /
Google Drive) instead of the gated Readest Cloud transfer queue.

Engine (services/sync/file/engine.ts, wire.ts):
- etag change-probe: one HEAD on library.json, cached per provider for the
  session. When the etag matches the last successful pull, reuse the cached
  index and skip both the index download and the discovery scan. An AUTH
  failure on the probe aborts the run like the full pull does.
- emptyDirs memo carried in the index: dirs found to hold no book file are
  recorded so clients stop re-listing them every run. Re-checked when the
  file arrives (uploadedHashes), on Full Sync, or when a legacy client drops
  the record; pruned only against a listing that actually ran.
- skip the index re-push when the rebuilt index is semantically identical to
  the pulled one, so a restamped byte-copy no longer churns the remote and
  invalidates peers' etag change detection.
- downloadBookFile() for the explicit per-book Download action.

Provider reuse (services/sync/file/providerRegistry.ts):
- memoise one provider per connection key, shared by every surface (reader
  per-book sync, library auto-sync, Sync now / pull to refresh). Reuses the
  Drive path->id cache instead of re-resolving /Readest, books/ and
  library.json by name query on every engine build. Drive connect/disconnect
  resets the cache since its token source changes identity with no key input
  changing.

Google Drive (services/sync/providers/gdrive/GoogleDriveProvider.ts):
- write fast-path: PATCH a path whose id is cached in place with no lookup,
  falling back to a full resolve on a 404 (stale id). Removes a files.list
  per PUT in the steady state.
- dev-only request diagnostics: one line per provider op and per HTTP attempt
  so a run's request budget can be attributed from the console.

Per-book transfers (services/sync/file/runLibrarySync.ts):
- runActiveFileBookUpload / runActiveFileBookDownload build the active
  provider's engine and push/pull a single book, stamping downloadedAt like
  the native path. Wired into the reader/library book actions with toasts.

UI, status, and i18n:
- Readest Cloud sub-page: drop the quota stats and wrap the "Account and
  Storage" row in the BoxedList primitive so it aligns to the design system.
- Shorten provider status/toast copy ("Active", "Google Drive session
  expired", "Library sync via {{provider}}", "KOReader") and translate the
  new keys across all locales.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 22:49:35 +02:00
..