f3c92f80d9abe1f29bf2e7f373411a67c4ae9c1f
15 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 loveheaven
|
675ee78bc9 |
perf(library): in-place re-import is a no-op on the same file path (#4597)
Re-importing a folder via the in-place option used to reopen, parse,
and partial-MD5 every file before its byHash entry could short-circuit
the import. Worse, the byHash short-circuit treated every hit as "user
dropped a fresh file matching a known book", so it refreshed
createdAt/updatedAt/downloadedAt and cleared filePath/deletedAt. For a
user re-scanning the same external folder, that quietly rewrote sort
order and wiped soft-delete state. And once the import returned, the
ingest pipeline still ran group / tag / upload work — including a
path-derived empty groupId that silently clobbered manual
GroupingModal assignments on every re-scan.
This change adds an explicit byFilePath fast path at the top of
`ingestFile` so a re-scan returns the existing library entry verbatim,
before any I/O AND before any downstream side effect:
byFilePath hit -> the same on-disk source is being re-scanned in
place; the right answer is no-op. Don't open the
file, don't touch any timestamps, don't re-cover,
don't run the group / tag / upload steps.
byHash hit -> a different source path resolves to a known book
(e.g. the user dropped a copy from elsewhere, or
a soft-deleted book is being revived); the
existing "refresh timestamps + clear deletedAt"
behavior in importBook is correct here and is
left intact.
Implementation:
- BookLookupIndex carries byHash / byMetaKey / byFilePath, with
byFilePath built only from non-deleted books that have an absolute
filePath. normalizeFilePathForIndex is the shared key function so
shouldImportInPlace and the index agree on case-insensitive
filesystems (macOS / iOS / Windows). osPlatform threads through
buildBookLookupIndex and BaseAppService.importBook so the renderer
and the importer compute the same key for the same file.
- ingestService.ingestFile gains a byFilePath fast path before its
importBook call: when `inPlace` was decided positive, the source
is a real on-disk path (not a PSE stream / URL / content URI),
and lookupIndex.byFilePath has a hit, return the existing Book
directly. Returning here — rather than inside importBook — is
deliberate: it skips the downstream group / tag / upload steps so
a re-scan can never silently overwrite a manual GroupingModal
assignment via a path-derived empty groupId.
- ingest-service.test.ts covers both halves: an in-place re-import
short-circuits importBook entirely (no call, existing object
returned, createdAt / updatedAt / groupId / groupName all
untouched); a copy-mode import (no external library folders) with
the same byFilePath entry still goes through importBook so dedup
falls back to byHash. import-metahash.test.ts retains the
BookLookupIndex builder test that deleted and url-backed books
are excluded from byFilePath.
Net effect: re-importing a folder of N already-imported books does
zero file opens, zero parses, zero MD5 passes, and leaves every book's
groupId / createdAt / deletedAt / cover untouched.
|
||
|
loveheaven
|
11d796361e |
perf(import+open): native Rust EPUB/MOBI parser, OPF prefetch, parallel TOC enrichment (#4369)
* perf(epub): add native EPUB parser in Rust
Introduce a Rust-side EPUB pre-parser exposing three Tauri commands:
* parse_epub_metadata - title/author/cover + partialMD5 in one
shot, for the import hot path
* parse_epub_full - OPF + nav.xhtml + toc.ncx bytes plus a
manifest size table, for the reader open
hot path
* extract_epub_cover_full - full-resolution cover bytes, for the
lock-screen wallpaper writer
All three avoid ferrying multi-MB blobs across the JS<->Rust IPC
boundary. Cover bytes returned by parse_epub_metadata are downscaled
to a webview-friendly JPEG when the long edge exceeds the library
thumbnail size.
No JS callers yet -- wired up in the following commits.
* perf(import): use native EPUB parser and downscale covers on Tauri targets
On Tauri (desktop/iOS/Android), importBook now forwards EPUB
metadata + cover extraction to the Rust parse_epub_metadata
command and reuses the partialMD5 it returns, skipping the
foliate-js full archive parse and the second pass over the file
for hashing.
As a side effect, the cover written to cover.png is downscaled
to a webview-friendly JPEG (long edge <= 512px), shrinking the
on-disk thumbnail from multi-MB to ~30-60KB per book. To keep
the lock-screen wallpaper feature unchanged, useAutoSaveBookCover
now pulls the original full-resolution cover via the Rust
extract_epub_cover_full command instead of copying the (now
downscaled) cover.png; falls back to the thumbnail when the
native path is unavailable.
Web targets and non-EPUB formats keep the existing path.
* perf(reader): prefetch EPUB OPF/nav from Rust on book open
When opening an EPUB on Tauri targets, DocumentLoader now calls the
Rust parse_epub_full command up-front to pull the OPF, EPUB3 nav,
NCX and the central-directory size map in a single IPC. The
foliate-js zip loader is wrapped so that loadText() of these
entries (and a synthetic META-INF/container.xml) is served from
that in-memory cache without inflating through zip.js, while
all other assets keep flowing through the original loader.
A small in-flight dedupe is added to the spine-text loader so the
nav pipeline (loadText + createDocument back-to-back on the same
href) doesn't pay for two zip.js inflate calls per chapter on
first open.
Reader store / app service plumbing: readerStore.openBook now
resolves an absolute on-disk path via the new
appService.resolveNativeBookFilePath / bookService.resolveNativeBookFilePath
helper and threads it into DocumentLoader as nativeFilePath so
the prefetch can fire. Web targets, non-EPUB formats and books
without a managed/external on-disk path skip the prefetch and
take the original code path.
* perf(nav): parallelize section scans and memoize fragment lookups
computeBookNav now processes sections via Promise.all instead of
a sequential for-loop, and within each section issues loadText()
and createDocument() concurrently. Combined with the in-flight
loadText dedupe added to the zip loader, each chapter pays for a
single zip inflate per nav build, and the inflates of different
chapters overlap.
enrichTocFromNavElements is restructured into two concurrent
phases: a cheap '<nav' substring filter on the inflated text, and
a parsed-document walk for the survivors. Most chapters fall out
in phase 1 without ever being parsed.
In fragments.ts, calculateFragmentSize now consults a
per-section position cache (makeFragmentPositionCache) so the
N-fragment loop is O(N) over the chapter HTML instead of O(N²).
A small isCfiAddressable guard is added to skip elements that
foliate-js's CFI generator can't address (documentElement, body
itself, detached nodes, nodes outside <body>) — these previously
threw and spammed console.warn for every fragment, now they
silently fall back to the section CFI.
* perf(import): use native MOBI/AZW/AZW3 parser on Tauri targets
On Tauri (desktop/iOS/Android), importBook now forwards
MOBI/AZW/AZW3/PRC metadata + cover extraction to the Rust
parse_mobi_metadata command and reuses the partialMD5 it returns,
skipping the foliate-js full-buffer parse and the second pass over
the file for hashing. Mirrors the existing EPUB native fast-path
added in e3fc4767 — bookService tries EPUB first, then MOBI; both
bridges fall back to the foliate-js DocumentLoader when the native
path is unavailable (web target, parse error, format mismatch).
The new mobi_parser is built on the mobi crate (KF7+KF8 reader,
zero JS-side touch). It reads title, author, publisher, ISBN, ASIN,
publish date, language, subjects and description from the MobiHeader
+ EXTH records, resolves the EXTH 201 cover offset against the PDB
image-record table (with ThumbOffset / first-image fallbacks), and
strips KindleGen's HTML wrapping in EXTH 103 so the description goes
into the library DB as plain text. The parsed cover is funneled
through the same maybe_resize_cover path as EPUB, so MOBI library
thumbnails are also clamped to a 512px-long-edge JPEG.
Cover-resize / partialMD5 / RawCoverImage are extracted into a new
parser_common module shared between epub_parser and mobi_parser, so
a single tweak (e.g. raising the thumbnail target) applies to every
native importer and the partialMD5 implementation can't drift between
the two paths (a divergent algorithm would silently re-import every
existing book under a new hash on the first run).
Web targets and non-Kindle formats keep the existing path.
* test(tauri): verify native Rust EPUB parser parity with foliate-js
Add a Tauri WebView parity suite (epub-parser-parity.tauri.test.ts) that
cross-checks the native Rust parser against foliate-js on the same fixtures:
parse_epub_metadata / parse_epub_full (title, author, language, identifier,
publisher, published, subjects, partialMD5, OPF + per-entry size table), and
that opening with the native prefetch produces the same BookDoc and
computeBookNav (TOC) output as the pure-JS path.
Fix a parity divergence the suite caught: the Rust OPF parser mapped
dcterms:modified onto `published`, but foliate-js keeps them separate and
leaves `published` empty -- so EPUB3 books carrying only the mandatory
dcterms:modified got a bogus publication date on the native import path. Map
only dc:date now; add regression tests.
Test infra:
- vitest.tauri.config.mts: add optimizeDeps (mirroring vitest.browser.config)
so foliate-js-importing tauri tests load -- otherwise esbuild's dep scan
can't resolve '@pdfjs/pdf.min.mjs', pre-bundling is skipped, and the CJS
deps fail to import ("Importing a module script failed").
- capabilities-extra/webdriver.json: fix __test__ -> __tests__ fs scope typo
so import tests can open fixtures under src/__tests__/.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(import): foliate-js owns EPUB/MOBI metadata via standalone extractors
Rust contributes only the mechanical work that's expensive on a
WebView — partialMD5, the downscaled cover, and (for EPUB) the raw
OPF bytes Rust already had to read for cover resolution. Metadata
extraction is delegated to foliate-js's two new standalone entry
points (`parseEpubMetadataFromXML`, `readMobiMetadata`) so the
import-path BookDoc and the reader-path BookDoc share a single
parser implementation.
EPUB
- `parse_epub_metadata` returns
`{ partialMd5, cover, coverMime, opfPath, opfBytes }`. OPF bytes
are a free byproduct of the cover-resolution scan.
- `tryNativeParseEpub` runs `parseEpubMetadataFromXML` on the OPF
bytes and assembles a lightweight BookDoc stub (metadata +
getCover). The importer doesn't drive `DocumentLoader.open()`, so
no zip central-directory scan, no nav/ncx inflate, no spine walk.
- `coverMime` is preserved so `bookService.importBook`'s
`cover.type === 'image/svg+xml'` branch still routes SVG covers
through svg2png.
MOBI / AZW / AZW3 / PRC
- `parse_mobi_metadata` returns `{ partialMd5, cover, coverMime }`.
`tryNativeParseMobi` runs foliate's `readMobiMetadata` on the
same File, which uses `MOBI.open(file, { metadataOnly: true })`
to parse PalmDB + MobiHeader + EXTH and short-circuit before the
MOBI6 / KF8 init() that walks every text record.
- `Book.metadata.identifier` is foliate's `mobi.uid.toString()`
(PalmDB UID), the canonical MOBI identifier the reader path uses.
bookService.importBook
- EPUB and MOBI native branches consume the bridge's BookDoc stub
directly. The stub's `getCover()` returns the Rust-downscaled
blob, falling back to foliate's own `getCover` thunk when Rust
didn't extract a cover.
Other
- Drop the unused `base64` Rust dependency: cover bytes go over IPC
as `Vec<u8>` (Tauri 2 transports them natively, like opfBytes /
navBytes / ncxBytes).
- Drop the `nativePrefetch` option on `DocumentLoaderOptions`; no
caller passes it. `nativeFilePath` keeps driving `parse_epub_full`
on the open hot path.
Tests
- vitest.tauri parity test asserts byte-equal partialMD5, cover
presence parity, OPF bytes that decode to a real `<package>`
document, and that `parseEpubMetadataFromXML` on those bytes
produces the same user-visible metadata fields (title / author /
language / identifier / published) as `DocumentLoader.open()`.
* test(tauri): add War and Peace MOBI fixture for native parser parity
The .tauri parser-parity suite previously had no .mobi/.azw3 asset, so the native MOBI parser (metadata + EXTH cover resolution) was uncovered. Adds a real KF8 MOBI ("War and Peace") to enable MOBI parity coverage against foliate-js.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(foliate-js): bump submodule to readest/foliate-js main (91191ca)
Replaces the ad-hoc 02f435a with the merged main commit 91191ca, which lands the standalone OPF/MOBI metadata extractors (parseEpubMetadataFromXML, readMobiMetadata) the import fast-path depends on (foliate#19), plus the RTL multi-view rect-mapper fix (foliate#20). The extractor code is byte-identical to 02f435a, so the bridges are unaffected.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Huang Xin <chrox.huang@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
loveheaven
|
ff605e000d |
feat(library): in-place import from registered external folders (#4315)
* feat(library): in-place import with cloud sync and symmetric local delete
Adds an `inPlace` option to importBook so a source file inside a
registered external library folder is referenced directly via
`book.filePath` instead of being copied into Books/<hash>/. Sidecars
(cover, config, nav) still live under Books/<hash>/.
ingestService routes through shouldImportInPlace, which marks an
import in-place when the absolute source path lives under any of
`settings.externalLibraryFolders` and is NOT inside a per-root
`Books/` subtree. The Readest data dir (`customRootDir`) is
intentionally excluded — that directory is Readest's home and
should freely hold hash copies; in-place is for user-registered
roots (Duokan, Calibre, Moon+ Reader, an iCloud mirror, …).
Cloud sync treats in-place books as first-class:
- uploadBook reads bytes from (book.filePath, 'None') when set.
The cloud key is unchanged, so a peer downloading the book
lands it under Books/<hash>/ as a normal hash copy.
- useBooksSync strips `book.filePath` before pushing — it is a
device-local path that is meaningless on any other device.
- ingestService no longer skips upload for in-place books;
autoUpload / forceUpload behave like any other book. Only
transient imports opt out.
- deleteBook 'local'/'both' now physically removes the source
file at book.filePath (base 'None'). Local-delete semantics
are symmetric with hash-copy books: the local copy is gone,
the cloud backup remains, a future pull restores under
Books/<hash>/. removeFile errors are swallowed.
New `SystemSettings.externalLibraryFolders?: string[]` (no UI yet;
registration entrypoint lands in a follow-up). Added to
BACKUP_SETTINGS_BLACKLIST alongside `localBooksDir` /
`customRootDir` so device-local paths don't ride cloud backups.
Tests: cloud-service, ingest-service, and backup-settings suites
cover in-place delete, multi-root matching, per-root `Books/`
guard, and the backup-strip.
* feat(library): one-tap "read in place" toggle in folder import
Surface the in-place / copy choice as a single "Read books in place (don't copy)" checkbox in the Import-from-Folder dialog. When the user opts in, the chosen directory is registered in `settings.externalLibraryFolders` and ingestService's `shouldImportInPlace` will route the books straight to importBook with `inPlace: true` — no copy into Books/<hash>/, sync still works, local delete still removes the source file (the symmetry was set up in the previous in-place commit).
User experience:
- First-time users hit the toggle once per library folder. The choice is also persisted to localStorage so subsequent dialog opens default to whatever they picked last.
- Repeat imports from a folder that's already registered as an external library folder force the toggle ON and disable it, with a help line explaining that imports from this folder are always in-place. The check is exact-string (after path normalization) so registering /Users/me/Duokan only locks the toggle for that exact path — picking /Users/me/Downloads after Duokan still shows the toggle in its normal state.
- URL-ingress / drag-drop replays go through `runFolderImport` without the dialog and default `readInPlace: false`. They still benefit from in-place automatically when the dropped path lives under an already-registered root, because that decision is made by `shouldImportInPlace` based on settings, not by the dialog flag.
Mechanics:
- ImportFromFolderResult gains `readInPlace: boolean`. ImportFromFolderDialog gains an `initialReadInPlace` prop (seeded from the new `readest:lastImportFolderReadInPlace` localStorage key) and an `isRegisteredExternalRoot` predicate it uses to render the locked / unlocked toggle.
- runFolderImport calls a new `registerExternalLibraryFolder` helper that appends the chosen directory to `settings.externalLibraryFolders` and persists settings, but only when `result.readInPlace` is true. `isRegisteredExternalRoot` does the inverse lookup the dialog needs. Both helpers normalize paths the same way `shouldImportInPlace` does so the predicate matches the ingest layer.
- The new feature has no effect for users who never flip the toggle: `externalLibraryFolders` stays empty, the path-prefix check in `shouldImportInPlace` returns false for every import, and books continue to be copied into Books/<hash>/ exactly as before.
Self-healing for externally-removed in-place books:
Once the dialog lets users opt their library into in-place mode, the source file becomes a piece of state Readest doesn't control — another app may rewrite it (e.g. Duokan persisting reading progress into the epub), the user may move it in Finder, or an external drive may unmount between sessions. Previously, clicking such a book would navigate into the reader, fail inside loadBookContent's `fs.openFile(book.filePath, 'None')` with a low-level IO error, flash an "Unable to open book" toast, and auto-bounce back to the library — leaving the stale library record in place so the next tap reproduces the same dance.
BookshelfItem.handleBookClick now probes availability before navigating, but only for purely-local in-place books (`book.filePath && !book.uploadedAt && !book.deletedAt`). If `appService.isBookAvailable` returns false — which for in-place books means the recorded `book.filePath` no longer exists at the OS level — we dispatch `delete-books` for that hash and show an info toast explaining the removal, instead of opening the reader.
Scope is intentionally narrow:
- Cloud-synced books still flow through `makeBookAvailable`'s on-demand download path; missing local copies trigger a re-download, not a deletion.
- Hash-copy books (no `filePath` set) are not probed: a missing Books/<hash>/ file under normal use signals a bug or filesystem corruption, not user intent, and silently dropping the record would hide the real problem.
- The dispatched delete-books event reuses the existing Bookshelf deletion path, so sidecar metadata and selection state are cleaned up the same way as a user-initiated delete. For in-place books that path doesn't touch any file outside Books/<hash>/, so the now-missing source location (or whatever the user did with it externally) is left alone — symmetric with 165f15a6.
* fix(library): centralize book content resolution
---------
Co-authored-by: Huang Xin <chrox.huang@gmail.com>
|
||
|
Huang Xin
|
5e366018df |
fix(cbz): ComicInfo metadata + CBZ page count + WebDAV i18n (#4282)
* fix(cbz,i18n): ComicInfo metadata + CBZ page count + WebDAV i18n Closes #4253 (ComicInfo.xml not read) and #4255 (CBZ shows "1 page left"). CBZ / ComicInfo (foliate-js submodule + Readest derivation): - comic-book.js: find ComicInfo.xml in subdirectories too, parse description / subject / identifier / published / series fields beyond the prior name+position pair. Series Count populates the canonical `belongsTo.series.total`; no top-level duplication. - bookService.ts / readerStore.ts: derive `metadata.seriesTotal` from `belongsTo.series.total` in parallel to the existing series / seriesIndex derivation. - ProgressBar / FooterBar / DesktopFooterBar: drop the hard-coded `pagesLeft = 1` for fixed-layout books and compute it from `section.total - section.current`. FooterBar uses `FIXED_LAYOUT_FORMATS.has(bookFormat)` so CBZ picks `section` (correct image count) instead of `pageinfo` (locations). - ProgressBar: switch the remaining-pages text to "in book" for fixed-layout titles (no chapter structure) and keep "in chapter" for reflowable books. WebDAV refactor for translation coverage: - WebDAVBrowsePane / SyncHistoryPanel called `t(...)` (passed as a prop) instead of `_(...)`. The i18next-scanner only looks for `_`, so ~53 strings were unreachable and shipped in English to every locale. Switched both components to call `useTranslation()` themselves; helpers that aren't React FCs take `_: TranslationFunc` so the scanner sees the literal calls. - WebDAVClient.checkConnection now returns a `code` discriminator (`SERVER_URL_REQUIRED` / `AUTH_FAILED` / `ROOT_NOT_FOUND` / `UNEXPECTED_STATUS` / `NETWORK`); raw English `message` is reserved for the dev console. New `formatConnectError` and `formatSyncError` helpers in WebDAVForm translate via a switch where each branch is a literal `_('...')`. Same treatment for the sync-failure path that previously surfaced raw e.message. - "Syncing 0 / {{total}}" is now parameterized as "Syncing {{n}} / {{total}}" with n=0 at startup so the digit formats naturally and the template can be reused mid-sync. - "Cleanup · {{count}} book(s)" hard-coded options used unsupported ternary; rewrote as plural-aware key. i18n scanner fix (i18next-scanner.config.cjs): - vinyl-fs walked into directories whose names end in source-file extensions (Next.js route folder `runtime-config.js/`, Playwright screenshot folder `*.test.tsx/`) and crashed with EISDIR. Resolved by expanding globs via `fs.globSync` and filtering to files only before handing to the scanner. TypeScript-syntax sites that broke esprima during extraction: - WebDAVBrowsePane / WebDAVForm: `(e as Error).message` and `failed[0]!.title` inside `_(..., options)` arguments. Replaced with `e instanceof Error ? e.message : String(e)` and `failed[0]?.title ?? ''` — also runtime-safer. User-facing em-dash cleanup: - Removed em-dashes from translation keys across SyncHistoryPanel / WebDAVForm / WebDAVBrowsePane / SyncPassphraseSection / send/page / replicaCryptoMiddleware / AIPanel. Tagline in `layout.tsx` kept. Locale translations: - ~2400 translations applied across all 33 locales for the keys that were either newly extractable, freshly worded, or pre-existing but untranslated. Zero `__STRING_NOT_TRANSLATED__` remain after the run. Misc: - next.config.mjs: drop `eslint.ignoreDuringBuilds: true` so build runs the same lint as CI. - Collection type: add `total?: string` for ComicInfo series count. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * ci(test): fix vitest invocation, run with 4 workers `pnpm test:pr:web` was chaining `pnpm test -- --watch=false`, which pnpm expanded into: dotenv -e .env -e .env.test.local -- vitest -- --watch=false The second `--` made vitest treat `--watch=false` as a positional file pattern, not a flag. Vitest then fell back to defaults (in CI's non-TTY env that still meant a one-shot run, so the suite passed), but the worker pool was effectively serialized for big chunks of the 243-file run — wall ~90 s on a 4-vCPU runner where the parallel-sum of phases was ~236 s (≈2.6× effective parallelism). Replace the chained pnpm invocation with a direct call to `vitest run --maxWorkers=4`, matching the 4 vCPUs the GH Actions ubuntu-latest runner provides. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
0fba5b7054 | feat(config): version book config schema (#4208) | ||
|
Huang Xin
|
981579c255 |
feat(sync): cross-device custom font sync (#4077)
* refactor(sync): kind-agnostic replica primitives Extract dict-only sync into shared primitives (registry, pull/apply orchestrator, persist env, schema allowlist) so other kinds can plug in. Companion changes: per-replica Storage Manager grouping, useReplicaPull boot-race recovery, manifest=null reconciliation on every boot pull, copyFile takes explicit srcBase + dstBase, settled- event helpers, lenient webDownload Content-Length (R2/S3 signed URLs commonly omit it), and generic "File" transfer toast copy any replica kind can share. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(sync): cross-device custom font sync Plug the font replica adapter into the kind-agnostic primitives: font store gains replica wiring, custom font import publishes the replica row + queues a binary upload, and bootstrap registers the font adapter and download-complete handler. Includes legacy flat-path migration so pre-existing fonts sync without re-import, full @font-face activation on auto-download (load + mount the rule, mirroring manual import), and a fix to createCustomFont so contentId / bundleDir / byteSize survive the trip through addFont — otherwise import-time publish silently no-oped on missing contentId. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Zeedif
|
ca8f0fe9f6 |
feat(opds): add OPDS-PSE streaming support and custom OPDS 2.0 parser (#3951)
* feat(opds): add OPDS-PSE streaming support and custom OPDS 2.0 parser * refactor(opds): remove custom parser, use updated foliate-js dependency * fix(opds): resolve PSE auth at fetch time, drop credentials from book.url Previously the streaming `pse://` virtual file baked the proxy URL with the basic auth header into `book.url`, which (a) failed on desktop where no proxy is used because the auth header was never applied to the page fetch, and (b) leaked the credential to the sync server because transient books still get pushed on first sync. Now the `pse://` payload stores only the upstream OPDS template URL plus the catalog id. A new `createPseStreamPageLoader` looks up the catalog from settings on each open, probes auth once (cached for the session), and applies the auth header via `tauriFetch` on desktop or via the proxy URL on web. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Huang Xin <chrox.huang@gmail.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
3e292af990 | refactor(nav): refactor book nav service with TOC enrichment (#3874) | ||
|
Huang Xin
|
b0cc5461af |
refactor(toc): cache navigable structure per book (#3869)
* refactor(toc): cache TOC + section fragments per book
Moves the TOC regrouping and section-fragment computation out of
foliate-js/epub.js #updateSubItems into the readest client as
computeBookNav / hydrateBookNav in utils/toc.ts. The result is
persisted to Books/{hash}/nav.json — capturing the book's full
navigable structure (TOC hierarchy + sections with hierarchical
fragments). Compute once, persist locally, hydrate on subsequent
opens. Designed to serve current human-facing navigation (TOC
sidebar, progress math) and future agentic navigation (LLM-driven
seeking by structural location).
Versioned by BOOK_NAV_VERSION for forward invalidation. Existing
books regenerate transparently on next open.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: update worktree scripts
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
Lex Moulton
|
030a7c0823 |
perf: optimize library operations for large collections (#3827)
* perf(store): decouple page turn from full library rewrite for large collections Previously every page turn triggered setLibrary() which copied the entire library array, ran refreshGroups() with MD5 hashing over all books, and caused cascading re-renders. With ~2800 books this made reading unusable. - Add hash-indexed Map to libraryStore for O(1) book lookups - Add lightweight updateBookProgress() that skips array copy and refreshGroups - Use hash index in setProgress, saveConfig, and initViewState - Batch cover URL generation with concurrency limit on library load Addresses #3714 * perf(import): replace filter()[0] with find() to short-circuit on first match * fix(store): replace Object.assign state mutation with immutable spread in setConfig * perf(persistence): remove JSON pretty-printing to reduce serialization overhead * fix(reader): stabilize debounce reference in useIframeEvents to prevent timer reset on re-render * perf(context): memoize provider values to prevent unnecessary consumer re-renders * perf(store): cache visible library to avoid refiltering on every access * perf(library): remove redundant refreshGroups call already triggered by setLibrary * perf(import): replace O(n) splice(0,0) with O(1) push for new book insertion * perf(import): defer library persistence to end of import batch instead of every 4 books * perf(library): skip full library reload on reader close since store is already in sync * fix: address PR review feedback for library perf optimizations Correctness fixes for issues found in code review: - fix(library): restore library reload on close-reader-window. Reader windows are independent Tauri webviews with their own libraryStore instance, so progress / readingStatus / move-to-front updates from the reader do not propagate to the main window. Reload from disk so the library reflects the changes the reader just persisted. - perf(import): wire BookLookupIndex into importBooks. The lookupIndex parameter on bookService.importBook had no caller, leaving the Map-based dedup path dead. Build the index once per import batch in app/library/page.tsx and thread it through appService.importBook so the O(1) dedup path is actually exercised. - perf(import): defer library save to end of batch. Add a skipSave option to libraryStore.updateBooks and call appService.saveLibraryBooks once after the entire import loop, instead of once per concurrency-4 sub-batch. - fix(store): make updateBookProgress immutable. The previous in-place mutation reused the same library array reference, bypassing Zustand change detection AND leaving the visibleLibrary cache holding stale Book references. Now slice the array, update the entry, and refresh visibleLibrary. Also make readingStatus a required parameter so future callers cannot accidentally clear it by omitting the argument. - fix(store): make saveConfig immutable. It previously mutated the Book object's progress / timestamps in place and used splice/unshift on the shared library array. Now spread to a new book object and rebuild via setLibrary. Also corrects the interface signature to return Promise<void> (the implementation was already async). - fix(store): make updateBook immutable for the same reason — it was mutating the previous-state library array before spreading. - fix(context): wrap AuthContext login/logout/refresh in useCallback. Without this, the useMemo deps array changed every render and the memo was a no-op, defeating the optimization the PR was trying to add. - fix(reader): use a ref for handlePageFlip in useMouseEvent's debounce. The empty-deps useMemo froze the first-render handler; with the ref the debounced wrapper always invokes the latest closure. Test coverage added: - library-store: immutable updateBookProgress, visibleLibrary cache refresh, deleted-book filtering, updateBooks skipSave option - book-data-store: immutable saveConfig, move-to-front correctness, visibleLibrary order, persistence behavior - import-metahash: BookLookupIndex update on new import, lookup-index consultation before scanning books array - auth-context (new file): context value identity stability across re-renders, callback identity stability - useIframeEvents (new file): debounced wheel handler dispatches to the latest handlePageFlip after re-render Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(types): move BookLookupIndex to types/book.ts Avoids the inline `import('@/services/bookService').BookLookupIndex` type annotation in types/system.ts. Both the AppService interface and the bookService implementation now import BookLookupIndex from the canonical location alongside Book. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(import): convert importBook params to options object Replace the long positional-argument list on appService.importBook (saveBook, saveCover, overwrite, transient, lookupIndex) with a single options object so callers no longer need to pad with `undefined, undefined, undefined, undefined` to reach the parameter they actually want to set. Before: await appService.importBook(file, library, undefined, undefined, undefined, undefined, lookupIndex); After: await appService.importBook(file, library, { lookupIndex }); The underlying bookService.importBook is also refactored to take an options object: required AppService callbacks (saveBookConfig, generateCoverImageUrl) are bundled with the optional flags via an ImportBookInternalOptions interface that extends the public ImportBookOptions defined in types/book.ts. All existing call sites updated to the new shape. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Huang Xin <chrox.huang@gmail.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
AK Venugopal
|
80cab8e56d |
feat: Hardcover.app Sync (#3724)
* feat(hardcover): add one-way Hardcover sync integration - Add HardcoverClient with ISBN/title-based book lookup, progress push, and note sync - Add HardcoverSyncMapStore for persistent local mapping of note IDs to Hardcover journal IDs - Add GraphQL queries/mutations for Hardcover API (insert/update/recreate journal entries) - Add DB migration for hardcover_note_mappings table (schema: hardcover-sync) - Add HardcoverSettings dialog component (connect/disconnect/enable toggle) - Add useHardcoverSync hook wired in Annotator for push-notes and push-progress events - Add Hardcover Sync section in BookMenu with per-book toggle (persisted in BookConfig) - Add HardcoverSettings type and DEFAULT_HARDCOVER_SETTINGS to system settings - Add hardcoverSyncEnabled per-book flag to BookConfig - Mount HardcoverSettingsWindow in Reader alongside KOSync and Readwise windows Sync is one-way (Readest → Hardcover), manual-action only, per-book opt-in. Supports idempotent note sync: insert new, skip unchanged, update changed, recreate on stale remote ID. * fix(hardcover): proxy API calls server-side to fix CORS, normalize Bearer token - Add /api/hardcover/graphql Next.js route that proxies POST requests server-side, bypassing CORS restrictions (Hardcover API has no Access-Control-Allow-Origin header) - On Tauri (desktop), calls Hardcover directly; on web, routes through the proxy - Normalize token in HardcoverClient: accept raw JWT or 'Bearer <jwt>' format - Update helper text to point to hardcover.app → Settings → API * fix(hardcover): surface note-sync no-ops and harden book resolution - Show toast feedback when book data is still loading, Hardcover is not configured, or the current book has no annotations/excerpts to sync - Show an explicit info toast when note sync finds no new changes - Parse Hardcover search results in the current hits/document response shape - Resolve note sync through ensureBookInLibrary for parity with progress sync - Add console logging for note/progress sync failures * debug(hardcover): add runtime instrumentation for note sync * feat(metadata): keep identifier stable and store ISBN separately - Add dedicated metadata.isbn field - Expose ISBN as its own editable field in book metadata - Preserve identifier semantics for existing source IDs and hashes - Route metadata auto-retrieval ISBN handling through the new field - Prefer metadata.isbn for Hardcover matching * fix(hardcover): avoid wasm sqlite for note mappings on web - Store Hardcover note sync mappings in localStorage on web - Keep sqlite-backed mappings for desktop/native environments - Remove the web-only database dependency from manual note sync * fix(hardcover): dedupe notes by payload hash across unstable note IDs - Add payload-hash lookup in HardcoverSyncMapStore - Reuse existing journal mapping when payload already synced - Prevent duplicate insertions when note IDs change or duplicate locally * fix(hardcover): avoid duplicate quote export when annotation note exists - Detect excerpt+annotation pairs for the same highlight - Skip standalone excerpt export when annotation has note text - Keep annotation export as the single source of truth * docs(hardcover): add consolidated change summary for review * fix(hardcover): suppress excerpt export by CFI when annotation note exists * fix(hardcover): suppress empty-note annotation duplicates when note exists * fix(hardcover): deduplicate notes by text and cfi base node * refactor(hardcover): optimize sync performance, add rate limiting and clean up debug tools * chore: remove dev-only change log from main * test(hardcover): add unit tests for sync mapping and client logic * chore: custom deployment and UI fixes * fix(hardcover): use timestamptz for accurate annotation time * fix(hardcover): use date scalar and RFC 3339 formatting for journal entries * Revert "chore: custom deployment and UI fixes" This reverts commit 0329aba7129d1e1ebf2c663804b8fba9a9f87b91. * Fix hardcover progress dates and surface imported ISBNs * Fix Hardcover currently reading sync * fix(hardcover): avoid promoting note sync status * style(hardcover): apply prettier formatting * test(hardcover): fix strict TypeScript assertions * test(hardcover): harden sync regression coverage * test(hardcover): fix lint and formatting regressions * fix(hardcover): narrow note dedupe range matching * refactor(hardcover): extract note dedupe helpers * refactor(isbn): extract metadata normalization helpers * feat(hardcover): improve synced quote formatting |
||
|
Huang Xin
|
7445d5befe |
feat(library): add batch refresh metadata option, closes #3294 (#3573)
Add a "Refresh Metadata" option under Advanced Settings that re-parses book files to update metadata (series info, language, etc.) for books imported before these fields were supported. - Add refreshBookMetadata() in bookService that opens a book file, re-parses with DocumentLoader, and updates metadata/metaHash/series without touching updatedAt or user-edited fields - Add menu item in SettingsMenu with progress display - Add unit tests for single-book metadata refresh - Translate new i18n strings across all 29 locales Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
91bc4ddec7 | feat(library): backup to and restore from a zip file (#3571) | ||
|
Huang Xin
|
db3dee025b | feat(booshelf): dedupe books with the same meta hash (#3535) | ||
|
Huang Xin
|
8274c6bc4f |
feat(node): refactor appService into focused service modules and add app service for Node runtime (#3530)
This also closes #3431. |