akai-tools/readest
2
0
Fork 0
forked from akai/readest
Code Pull Requests Activity
2,489 Commits 3 Branches 102 Tags
c8e2c953351dd0012ae0e643afee9f562efee76f
Commit Graph

9 Commits

Author SHA1 Message Date
Huang Xin c8e2c95335 feat(library): auto-import new books from watched folders (#3889) (#4902) 
* feat(library): add autoImportFromFolders setting (default off)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(library): add selectNewImportableFiles folder-scan filter

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(library): add useAutoImportFolders trigger hook

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(library): auto-import new books from watched folders on open and focus

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(library): add Auto Import New Books from Folders toggle

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(library): don't resurrect deleted books or re-toast bad files on folder auto-import

- Add collectKnownSourcePaths() pure helper that includes soft-deleted books
- importBooks() accepts { silent } option and returns failedPaths
- autoImportFromWatchedFolders uses collectKnownSourcePaths and session-scoped
  autoImportFailedPathsRef to skip already-failed files on subsequent scans

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(library): make folder auto-import a per-folder option in the import dialog

Replace the global autoImportFromFolders toggle (and its standalone Settings
menu entry) with a per-folder opt-in shown as a sub-option of Read in place in
the Import-from-Folder dialog. Store the watched set as settings.autoImportFolders
(a subset of externalLibraryFolders; device-local, backup-blacklisted). The
library rescan now iterates autoImportFolders instead of a global-gated
externalLibraryFolders.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-07-03 12:14:08 +02:00
Huang Xin 7e78f80e14 feat(sync): Google Drive cloud sync + premium Third-party Cloud Sync section (desktop) (#4821) 
* feat(sync): add Google Drive file-sync provider core

Second FileSyncProvider for the merged provider-agnostic file-sync engine,
behind the provider seam. This is the CI-testable core only: no settings UI
and no platform OAuth runners yet (those land in later phases).

- GoogleDriveProvider over the Drive v3 REST API: id-addressed path
  resolution with a per-instance id cache, create-then-name uploads, real
  idempotent ensureDir, files.list pagination, Retry-After-aware 429/5xx
  backoff, per-path folder-creation locks with deterministic duplicate
  collapse, stale-id eviction, and FileSyncError mapping (403 split into
  rate-limit vs permission).
- DI OAuth layer: pkce, parseRedirect (redirect-target + CSRF state),
  reverseDnsRedirect, tokenStore (iOS client, no secret), oauthFlow.
- PersistedDriveAuth with single-flight token refresh; keychain-backed
  token store with no ephemeral fallback for the refresh token; account
  label via about.get.
- providerRegistry (backend kind to provider) and buildGoogleDriveProvider
  assembly.
- Shared transport-agnostic provider semantic contract, run against both
  WebDAV and Drive.
- Keyed secure-KV bridge contract (set/get/clear_secure_item); the native
  keychain implementation lands with the desktop OAuth slice that first
  exercises it.

Adapted from ratatabananana-bit/Readest-google-drive-mod-patcher (AGPL-3.0)
with the author's explicit permission.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(sync): multi-provider file-sync settings + sync-state foundation

PR2 foundation for a second file-sync backend (Google Drive). The
behaviour-sensitive reader-hook and Sync-now form generalization land in
PR3 alongside OAuth, where Drive actually connects and the multi-provider
paths can be exercised and live-verified (and the extracted form gets its
second consumer, avoiding a single-use abstraction).

- GoogleDriveSettings type (mirrors WebDAVSettings minus URL/credentials/
  rootPath, plus accountLabel) wired into SystemSettings, with
  DEFAULT_GOOGLE_DRIVE_SETTINGS in the defaults.
- googleDrive.deviceId + googleDrive.lastSyncedAt added to the backup
  blacklist so device-local sync identity / cursors never restore onto
  another device. Covered by the existing backup-settings test.
- Generalize webdavSyncStore into fileSyncStore: per-backend progress keyed
  by provider kind, plus a global library-sync mutex (beginSync returns
  false when another backend already holds the lock) since every backend's
  syncLibrary mutates the same local library. Migrate WebDAVForm and
  IntegrationsPanel to the keyed API; WebDAV behaviour is unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(native-bridge): add keyed secure key-value store commands

A generic, keyed secret store over the same OS keychain backends as the
sync passphrase (set/get/clear_secure_item), so secrets that aren't the
single sync passphrase get the same XSS-free cross-launch persistence
without each needing its own native command. The Google Drive OAuth token
store (PR1's KeychainTokenPersistence) is the first consumer; a future
cloud provider's refresh token reuses it.

- Desktop (macOS/Windows/Linux): keyring-core, keyed by the item key as
  the entry account under the existing "Readest Safe Storage" service.
- Android: EncryptedSharedPreferences (a dedicated readest_secure_items_v1
  file, the item key as the pref key).
- iOS: Security framework Keychain (kSecClassGenericPassword, dedicated
  service, the item key as kSecAttrAccount).

Registered in the plugin invoke handler + build COMMANDS + default
permission set (autogenerated permission files regenerated; the passphrase
entries are preserved). The TS bridge wrappers shipped in PR1.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(sync): desktop Google Drive OAuth runner + connect flow

The desktop half of Drive sign-in: open consent in the system browser, capture
the reverse-DNS redirect the OS routes back, and exchange the code for tokens.

- oauthDesktop.ts: runDesktopDeepLinkOAuth wires the DI OAuth flow to the
  desktop mechanics (open default browser, capture via single-instance /
  onOpenUrl, cold-browser fallback after a grace period, hard deadline). Fully
  headless-unit-tested via injected deps.
- spawn_fresh_browser.rs (+ registration, Windows-only winreg dep): the cold
  browser the runner falls back to when the user's already-running browser
  snapshotted protocol associations before the scheme was registered (a
  Windows-specific failure). Resolves the default browser from the registry and
  spawns it cold with an isolated --user-data-dir; a no-op on macOS/Linux where
  the default-browser open already routes the redirect. Pure helpers unit-tested.
- connectGoogleDrive.ts: run the platform OAuth runner, persist the token
  (fail-loud — Drive is not reported connected if the refresh token does not
  save), and resolve the account label via about.get (best-effort).

OAuth runner adapted from ratatabananana-bit/Readest-google-drive-mod-patcher
(AGPL-3.0) with the author's permission. Scheme registration + the ingress
redirect filter + the Drive connect UI land in the following commits; live
desktop verification follows once the official Google client id is provisioned.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(sync): filter Google OAuth redirects out of the deep-link ingress

The reverse-DNS OAuth redirect (com.googleusercontent.apps.<id>:/oauthredirect)
is delivered through the same single-instance / onOpenUrl channels as book-file
deep links. Without a filter the book-import consumer would treat the redirect
URL as a file path to open. Drop it at the ingress source (useAppUrlIngress)
before the app-incoming-url broadcast, so no consumer ever sees it; the Drive
sign-in runner still captures it via its own listeners.

isGoogleOAuthRedirectUrl matches the scheme prefix (not a specific client id),
so it stays correct regardless of which client is baked into the build.

Note: registering the scheme in tauri.conf.json (so the OS routes it back to the
app) needs the official Google client id, which is a provisioning prerequisite.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(sync): bake the official Google Drive OAuth client id + redirect scheme

Provisioned the Readest Google Cloud OAuth client (iOS application type, no
secret, drive.file scope). Bake the client id as the default in
getGoogleClientId (overridable via NEXT_PUBLIC_GOOGLE_CLIENT_ID for forkers,
who must also regenerate the manifest schemes) and register the derived
reverse-DNS redirect scheme com.googleusercontent.apps.<id> in tauri.conf.json
(desktop + mobile deep-link) so the OS routes the OAuth redirect back to the
app. The client id is a public client identifier, not a secret.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(sync): Google Drive connect UI + shared FileSyncForm

Make Drive usable from Settings, and extract the now-two-consumer sync controls.

- FileSyncForm: the provider-agnostic sync controls (sub-toggles, conflict
  strategy, manual "Sync now" with progress + result toast), parameterised by
  backend kind and building the provider through the registry. Extracted from
  WebDAVForm now that a second consumer exists. WebDAVForm keeps its
  URL/credentials connect panel + browse pane and renders FileSyncForm for the
  sync section; behaviour is unchanged (WebDAV "Sync now" goes through the same
  provider via the registry).
- GoogleDriveForm: an OAuth connect panel (Connect -> runGoogleDriveConnect ->
  store token in keychain -> "Connected as <email>"; Disconnect) + FileSyncForm.
- googleDriveConnect.ts: assemble the env client id + keychain + desktop runner
  into connectGoogleDrive/disconnectGoogleDrive for the UI.
- IntegrationsPanel: a "Google Drive" row + sub-page, shown only on desktop
  (mobile OAuth runners land in later phases).

Reader-side auto-sync (generalizing useWebDAVSync) is a follow-up; manual
"Sync now" already exercises the full Drive stack. Full suite 6412 green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(settings): unified Third-party Cloud Sync section (exclusive provider)

Group WebDAV + Google Drive into a new "Third-party Cloud Sync" section and make
them mutually exclusive — only one cloud provider syncs the library at a time.

- New unified "Cloud Sync" sub-page (CloudSyncForm): a provider picker (radio,
  the AIPanel mutually-exclusive pattern) on top, the shared FileSyncForm sync
  options below for whichever provider is active. Google Drive is offered only on
  desktop; on mobile the page is WebDAV only and the picker is hidden.
- withActiveCloudProvider helper: enabling one provider disables the other in one
  save. Both panels' connect/activate paths use it. Unit-tested.
- WebDAVForm / GoogleDriveForm refactored into embeddable panels (the unified
  page owns the header). Drive gains a "configured but inactive" state so
  switching back re-activates it without a fresh sign-in; explicit Disconnect
  clears the keychain token.
- IntegrationsPanel: remove the two separate WebDAV / Google Drive rows from
  "Reading Sync" (now KOReader Sync / Readwise / Hardcover only); add the
  Third-party Cloud Sync section with one Cloud Sync row (status = active
  provider). Old webdav/gdrive deep-links route to the unified page.

Also removes the temporary Drive concurrency probe (the upload already runs at
the intended concurrency 4; the probe confirmed it).

Full suite 6416 green; lint + format clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(reader): auto-sync the active cloud provider while reading

Generalize the reader sync hook from useWebDAVSync to useFileSync so the active
third-party cloud provider (WebDAV OR Google Drive) syncs per-book while reading
— pull-on-open, debounced push on progress/booknote changes, cover/file upload —
not just via the manual "Sync now" in settings.

Since the providers are mutually exclusive, the hook drives exactly the one
enabled backend, built through the provider registry. The build is async (the
Google Drive provider probes the OS keychain), so the engine lives in state and
the pull-on-open waits for it; switching providers mid-session resets the
per-book locks. The engine is keyed on connection-relevant settings so a
lastSyncedAt write doesn't re-probe the keychain. deviceId / lastSyncedAt now
write the active provider's settings slice; the auth-failed toast is
provider-neutral; the per-book events are renamed *-file-sync.

WebDAV reader-sync behaviour is unchanged. Full suite 6416 green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(settings): surface cloud providers in the section with inline switch

Show WebDAV + Google Drive as separate rows in the Third-party Cloud Sync
section (instead of one "Cloud Sync" row), so both providers are visible and the
active one can be switched right there.

- CloudProviderRow: a trailing radio makes a provider the single active sync
  target inline (enabled only when it's already configured — WebDAV creds / a
  Drive token); the row body / chevron opens its config sub-page (connect, sync
  options, disconnect). Status reads Active / Configured / Not connected, with a
  Syncing… indicator.
- Each provider drills into its own sub-page again (WebDAV / Google Drive),
  rendering the embeddable panel under a SubPageHeader; the brief unified
  CloudSyncForm picker page is removed (its old deep-link maps to Google Drive).
- Switching stays exclusive via withActiveCloudProvider; an inline switch trusts
  the stored credentials/token (no re-validate / re-OAuth).

Full suite 6416 green; lint + format clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(sync): gate third-party cloud sync behind a premium plan

WebDAV + Google Drive sync is now a premium feature: available on any paid plan
(Plus, Pro, or Lifetime), not on free.

- isCloudSyncInPlan(plan) helper (mirrors isEmailInPlan; plus/pro/purchase).
- IntegrationsPanel: free users see the Third-party Cloud Sync section with an
  upgrade row ("Available on Plus, Pro, or Lifetime") that opens the plans page
  instead of the provider rows; the cloud-sync deep-links are gated too (waiting
  for the plan to load before deciding).
- useFileSync: the reader's auto-sync only runs on a paid plan, so a downgraded
  user's sync stops even if a provider's enabled flag lingers.

Full suite 6418 green; lint + format clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(sync): escape backslashes in Drive query literals (CodeQL)

escapeDriveLiteral escaped single quotes but not the backslash escape
character, so a file name containing a backslash (or ending in one) could
break out of the single-quoted Drive `files.list` query literal and malform
the query. Escape backslashes first, then single quotes, so the backslashes
added for the quotes are not doubled.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-27 10:28:08 +02:00
Huang Xin febb0d9a69 fix(backup): normalize Windows backslash paths in backup zip entries (#4706) 
A backup .zip exported on Windows failed to restore on every platform
(Web, Android, Windows): books restored with metadata but no files or
covers.

`appService.readDirectory` returns paths using the host separator, so on
Windows `file.path` is `hash\cover.png` (backslash). `addBackupEntriesToZip`
used that verbatim as the zip entry name, so entries were named with `\`.
Restore matches a book's files with `filename.startsWith(`${hash}/`)`
(forward slash), which never matched the backslash names, so every book
file was silently skipped.

Normalize the zip entry name to forward slashes when adding files. Entry
names are now cross-platform and restorable everywhere. Already-exported
broken backups need re-exporting from the fixed app.

Fixes #4703

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-21 18:48:07 +02:00
loveheaven ff605e000d feat(library): in-place import from registered external folders (#4315) 
* feat(library): in-place import with cloud sync and symmetric local delete

Adds an `inPlace` option to importBook so a source file inside a
registered external library folder is referenced directly via
`book.filePath` instead of being copied into Books/<hash>/. Sidecars
(cover, config, nav) still live under Books/<hash>/.

ingestService routes through shouldImportInPlace, which marks an
import in-place when the absolute source path lives under any of
`settings.externalLibraryFolders` and is NOT inside a per-root
`Books/` subtree. The Readest data dir (`customRootDir`) is
intentionally excluded — that directory is Readest's home and
should freely hold hash copies; in-place is for user-registered
roots (Duokan, Calibre, Moon+ Reader, an iCloud mirror, …).

Cloud sync treats in-place books as first-class:

  - uploadBook reads bytes from (book.filePath, 'None') when set.
    The cloud key is unchanged, so a peer downloading the book
    lands it under Books/<hash>/ as a normal hash copy.

  - useBooksSync strips `book.filePath` before pushing — it is a
    device-local path that is meaningless on any other device.

  - ingestService no longer skips upload for in-place books;
    autoUpload / forceUpload behave like any other book. Only
    transient imports opt out.

  - deleteBook 'local'/'both' now physically removes the source
    file at book.filePath (base 'None'). Local-delete semantics
    are symmetric with hash-copy books: the local copy is gone,
    the cloud backup remains, a future pull restores under
    Books/<hash>/. removeFile errors are swallowed.

New `SystemSettings.externalLibraryFolders?: string[]` (no UI yet;
registration entrypoint lands in a follow-up). Added to
BACKUP_SETTINGS_BLACKLIST alongside `localBooksDir` /
`customRootDir` so device-local paths don't ride cloud backups.

Tests: cloud-service, ingest-service, and backup-settings suites
cover in-place delete, multi-root matching, per-root `Books/`
guard, and the backup-strip.

* feat(library): one-tap "read in place" toggle in folder import

Surface the in-place / copy choice as a single "Read books in place (don't copy)" checkbox in the Import-from-Folder dialog. When the user opts in, the chosen directory is registered in `settings.externalLibraryFolders` and ingestService's `shouldImportInPlace` will route the books straight to importBook with `inPlace: true` — no copy into Books/<hash>/, sync still works, local delete still removes the source file (the symmetry was set up in the previous in-place commit).

User experience:

  - First-time users hit the toggle once per library folder. The choice is also persisted to localStorage so subsequent dialog opens default to whatever they picked last.

  - Repeat imports from a folder that's already registered as an external library folder force the toggle ON and disable it, with a help line explaining that imports from this folder are always in-place. The check is exact-string (after path normalization) so registering /Users/me/Duokan only locks the toggle for that exact path — picking /Users/me/Downloads after Duokan still shows the toggle in its normal state.

  - URL-ingress / drag-drop replays go through `runFolderImport` without the dialog and default `readInPlace: false`. They still benefit from in-place automatically when the dropped path lives under an already-registered root, because that decision is made by `shouldImportInPlace` based on settings, not by the dialog flag.

Mechanics:

  - ImportFromFolderResult gains `readInPlace: boolean`. ImportFromFolderDialog gains an `initialReadInPlace` prop (seeded from the new `readest:lastImportFolderReadInPlace` localStorage key) and an `isRegisteredExternalRoot` predicate it uses to render the locked / unlocked toggle.

  - runFolderImport calls a new `registerExternalLibraryFolder` helper that appends the chosen directory to `settings.externalLibraryFolders` and persists settings, but only when `result.readInPlace` is true. `isRegisteredExternalRoot` does the inverse lookup the dialog needs. Both helpers normalize paths the same way `shouldImportInPlace` does so the predicate matches the ingest layer.

  - The new feature has no effect for users who never flip the toggle: `externalLibraryFolders` stays empty, the path-prefix check in `shouldImportInPlace` returns false for every import, and books continue to be copied into Books/<hash>/ exactly as before.

Self-healing for externally-removed in-place books:

  Once the dialog lets users opt their library into in-place mode, the source file becomes a piece of state Readest doesn't control — another app may rewrite it (e.g. Duokan persisting reading progress into the epub), the user may move it in Finder, or an external drive may unmount between sessions. Previously, clicking such a book would navigate into the reader, fail inside loadBookContent's `fs.openFile(book.filePath, 'None')` with a low-level IO error, flash an "Unable to open book" toast, and auto-bounce back to the library — leaving the stale library record in place so the next tap reproduces the same dance.

  BookshelfItem.handleBookClick now probes availability before navigating, but only for purely-local in-place books (`book.filePath && !book.uploadedAt && !book.deletedAt`). If `appService.isBookAvailable` returns false — which for in-place books means the recorded `book.filePath` no longer exists at the OS level — we dispatch `delete-books` for that hash and show an info toast explaining the removal, instead of opening the reader.

  Scope is intentionally narrow:
  - Cloud-synced books still flow through `makeBookAvailable`'s on-demand download path; missing local copies trigger a re-download, not a deletion.
  - Hash-copy books (no `filePath` set) are not probed: a missing Books/<hash>/ file under normal use signals a bug or filesystem corruption, not user intent, and silently dropping the record would hide the real problem.
  - The dispatched delete-books event reuses the existing Bookshelf deletion path, so sidecar metadata and selection state are cleaned up the same way as a user-initiated delete. For in-place books that path doesn't touch any file outside Books/<hash>/, so the now-missing source location (or whatever the user did with it externally) is left alone — symmetric with 165f15a6.

* fix(library): centralize book content resolution

---------

Co-authored-by: Huang Xin <chrox.huang@gmail.com>
 2026-05-27 12:08:36 +02:00
loveheaven 98049282eb feat(ai): add OpenRouter provider and unify provider HTTP transport (#4289) 
* OpenRouter: new OpenAI-compatible provider with chat + embedding model
  routing, health check via /models, and a fetchOpenRouterModels() helper
  for the settings UI. API key, base URL and model fields are persisted in
  AISettings, surfaced in AIPanel, indexed by commandRegistry, and added
  to backupService's credential allow-list so the key round-trips through
  encrypted backups.

* utils/httpFetch: introduce getAIFetch() as the single decision point for
  outbound AI traffic. In Tauri it returns @tauri-apps/plugin-http's fetch
  (Rust/reqwest transport, no renderer CORS preflight, no Android cleartext
  block); on the web build it falls back to window.fetch. OllamaProvider
  is migrated end-to-end — both ai-sdk-ollama streaming and the /api/tags
  health probe — and the new OpenRouterProvider uses the same path, so any
  future provider only has to call getAIFetch().

* Tests: unit tests for OpenRouter provider behavior (model selection,
  availability, health check) and a backup-settings round-trip test
  ensuring openrouterApiKey is treated as a credential field.
 2026-05-25 18:32:07 +02:00
Huang Xin 52f9634810 feat(backup): include global settings in backup zip (#4211) 
* feat(backup): include global settings in backup zip

Backup zips previously held only book files and library.json. Issue
#4098 asks for app configuration to be backed up too.

A `settings.json` snapshot is now written at the zip root. Restore
deep-merges it onto the current device's settings, so fields the
snapshot omits keep their current values.

`sanitizeSettingsForBackup` strips, via a blacklist, fields that are
device-specific or sync/migration bookkeeping (filesystem paths,
replica/kosync device ids, sync cursors, lastOpenBooks, screen
brightness, schema versions). Account credentials (kosync/Readwise/
Hardcover tokens, AI gateway key, OPDS catalog logins) are stripped
unless the user opts in via a new "Include account credentials"
checkbox in the Backup & Restore dialog — the zip is unencrypted.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(backup): keep revived books visible after a cloud-synced restore

When the library is deleted (soft delete) and the deletion has synced
to the cloud, restoring an older backup un-deletes the books locally —
but the next sync's last-writer-wins merge re-applied the cloud's
deletion tombstone, so the restored books vanished again.

The deletion never bumps `updatedAt`, so a restored book and its cloud
tombstone share the same timestamp; `processOldBook` breaks the tie
toward the cloud.

`reviveRestoredBooks` now fixes up books that were soft-deleted locally
but present in the backup:

- Bumps `updatedAt` so the restore out-ranks the cloud tombstone. A
  single uniform offset is applied to every revived book, so their
  relative order — and the library's "Updated" sort — is preserved
  exactly; the newest maps to now, none land in the future.
- Clears `syncedAt` so the next push re-uploads them and corrects the
  cloud rows.
- Restores `downloadedAt` / `coverDownloadedAt` from the backup record
  (the local deletion had cleared them) so revived books are not shown
  as not-downloaded even though their files were re-extracted.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-17 21:16:46 +02:00
Huang Xin 0fba5b7054 feat(config): version book config schema (#4208) 2026-05-17 18:23:46 +02:00
Lex Moulton 030a7c0823 perf: optimize library operations for large collections (#3827) 
* perf(store): decouple page turn from full library rewrite for large collections

Previously every page turn triggered setLibrary() which copied the entire
library array, ran refreshGroups() with MD5 hashing over all books, and
caused cascading re-renders. With ~2800 books this made reading unusable.

- Add hash-indexed Map to libraryStore for O(1) book lookups
- Add lightweight updateBookProgress() that skips array copy and refreshGroups
- Use hash index in setProgress, saveConfig, and initViewState
- Batch cover URL generation with concurrency limit on library load

Addresses #3714

* perf(import): replace filter()[0] with find() to short-circuit on first match

* fix(store): replace Object.assign state mutation with immutable spread in setConfig

* perf(persistence): remove JSON pretty-printing to reduce serialization overhead

* fix(reader): stabilize debounce reference in useIframeEvents to prevent timer reset on re-render

* perf(context): memoize provider values to prevent unnecessary consumer re-renders

* perf(store): cache visible library to avoid refiltering on every access

* perf(library): remove redundant refreshGroups call already triggered by setLibrary

* perf(import): replace O(n) splice(0,0) with O(1) push for new book insertion

* perf(import): defer library persistence to end of import batch instead of every 4 books

* perf(library): skip full library reload on reader close since store is already in sync

* fix: address PR review feedback for library perf optimizations

Correctness fixes for issues found in code review:

- fix(library): restore library reload on close-reader-window. Reader
  windows are independent Tauri webviews with their own libraryStore
  instance, so progress / readingStatus / move-to-front updates from
  the reader do not propagate to the main window. Reload from disk
  so the library reflects the changes the reader just persisted.

- perf(import): wire BookLookupIndex into importBooks. The lookupIndex
  parameter on bookService.importBook had no caller, leaving the
  Map-based dedup path dead. Build the index once per import batch
  in app/library/page.tsx and thread it through appService.importBook
  so the O(1) dedup path is actually exercised.

- perf(import): defer library save to end of batch. Add a skipSave
  option to libraryStore.updateBooks and call appService.saveLibraryBooks
  once after the entire import loop, instead of once per concurrency-4
  sub-batch.

- fix(store): make updateBookProgress immutable. The previous in-place
  mutation reused the same library array reference, bypassing Zustand
  change detection AND leaving the visibleLibrary cache holding stale
  Book references. Now slice the array, update the entry, and refresh
  visibleLibrary. Also make readingStatus a required parameter so
  future callers cannot accidentally clear it by omitting the argument.

- fix(store): make saveConfig immutable. It previously mutated the Book
  object's progress / timestamps in place and used splice/unshift on
  the shared library array. Now spread to a new book object and rebuild
  via setLibrary. Also corrects the interface signature to return
  Promise<void> (the implementation was already async).

- fix(store): make updateBook immutable for the same reason — it was
  mutating the previous-state library array before spreading.

- fix(context): wrap AuthContext login/logout/refresh in useCallback.
  Without this, the useMemo deps array changed every render and the
  memo was a no-op, defeating the optimization the PR was trying to
  add.

- fix(reader): use a ref for handlePageFlip in useMouseEvent's debounce.
  The empty-deps useMemo froze the first-render handler; with the ref
  the debounced wrapper always invokes the latest closure.

Test coverage added:
- library-store: immutable updateBookProgress, visibleLibrary cache
  refresh, deleted-book filtering, updateBooks skipSave option
- book-data-store: immutable saveConfig, move-to-front correctness,
  visibleLibrary order, persistence behavior
- import-metahash: BookLookupIndex update on new import, lookup-index
  consultation before scanning books array
- auth-context (new file): context value identity stability across
  re-renders, callback identity stability
- useIframeEvents (new file): debounced wheel handler dispatches to
  the latest handlePageFlip after re-render

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor(types): move BookLookupIndex to types/book.ts

Avoids the inline `import('@/services/bookService').BookLookupIndex`
type annotation in types/system.ts. Both the AppService interface and
the bookService implementation now import BookLookupIndex from the
canonical location alongside Book.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor(import): convert importBook params to options object

Replace the long positional-argument list on appService.importBook
(saveBook, saveCover, overwrite, transient, lookupIndex) with a
single options object so callers no longer need to pad with
`undefined, undefined, undefined, undefined` to reach the parameter
they actually want to set.

Before:
  await appService.importBook(file, library, undefined, undefined,
                              undefined, undefined, lookupIndex);

After:
  await appService.importBook(file, library, { lookupIndex });

The underlying bookService.importBook is also refactored to take an
options object: required AppService callbacks (saveBookConfig,
generateCoverImageUrl) are bundled with the optional flags via an
ImportBookInternalOptions interface that extends the public
ImportBookOptions defined in types/book.ts.

All existing call sites updated to the new shape.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Huang Xin <chrox.huang@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-11 16:32:35 +02:00
Huang Xin 91bc4ddec7 feat(library): backup to and restore from a zip file (#3571) 2026-03-20 18:27:52 +01:00