perf(import+open): native Rust EPUB/MOBI parser, OPF prefetch, parallel TOC enrichment (#4369)
* perf(epub): add native EPUB parser in Rust
Introduce a Rust-side EPUB pre-parser exposing three Tauri commands:
* parse_epub_metadata - title/author/cover + partialMD5 in one
shot, for the import hot path
* parse_epub_full - OPF + nav.xhtml + toc.ncx bytes plus a
manifest size table, for the reader open
hot path
* extract_epub_cover_full - full-resolution cover bytes, for the
lock-screen wallpaper writer
All three avoid ferrying multi-MB blobs across the JS<->Rust IPC
boundary. Cover bytes returned by parse_epub_metadata are downscaled
to a webview-friendly JPEG when the long edge exceeds the library
thumbnail size.
No JS callers yet -- wired up in the following commits.
* perf(import): use native EPUB parser and downscale covers on Tauri targets
On Tauri (desktop/iOS/Android), importBook now forwards EPUB
metadata + cover extraction to the Rust parse_epub_metadata
command and reuses the partialMD5 it returns, skipping the
foliate-js full archive parse and the second pass over the file
for hashing.
As a side effect, the cover written to cover.png is downscaled
to a webview-friendly JPEG (long edge <= 512px), shrinking the
on-disk thumbnail from multi-MB to ~30-60KB per book. To keep
the lock-screen wallpaper feature unchanged, useAutoSaveBookCover
now pulls the original full-resolution cover via the Rust
extract_epub_cover_full command instead of copying the (now
downscaled) cover.png; falls back to the thumbnail when the
native path is unavailable.
Web targets and non-EPUB formats keep the existing path.
* perf(reader): prefetch EPUB OPF/nav from Rust on book open
When opening an EPUB on Tauri targets, DocumentLoader now calls the
Rust parse_epub_full command up-front to pull the OPF, EPUB3 nav,
NCX and the central-directory size map in a single IPC. The
foliate-js zip loader is wrapped so that loadText() of these
entries (and a synthetic META-INF/container.xml) is served from
that in-memory cache without inflating through zip.js, while
all other assets keep flowing through the original loader.
A small in-flight dedupe is added to the spine-text loader so the
nav pipeline (loadText + createDocument back-to-back on the same
href) doesn't pay for two zip.js inflate calls per chapter on
first open.
Reader store / app service plumbing: readerStore.openBook now
resolves an absolute on-disk path via the new
appService.resolveNativeBookFilePath / bookService.resolveNativeBookFilePath
helper and threads it into DocumentLoader as nativeFilePath so
the prefetch can fire. Web targets, non-EPUB formats and books
without a managed/external on-disk path skip the prefetch and
take the original code path.
* perf(nav): parallelize section scans and memoize fragment lookups
computeBookNav now processes sections via Promise.all instead of
a sequential for-loop, and within each section issues loadText()
and createDocument() concurrently. Combined with the in-flight
loadText dedupe added to the zip loader, each chapter pays for a
single zip inflate per nav build, and the inflates of different
chapters overlap.
enrichTocFromNavElements is restructured into two concurrent
phases: a cheap '<nav' substring filter on the inflated text, and
a parsed-document walk for the survivors. Most chapters fall out
in phase 1 without ever being parsed.
In fragments.ts, calculateFragmentSize now consults a
per-section position cache (makeFragmentPositionCache) so the
N-fragment loop is O(N) over the chapter HTML instead of O(N²).
A small isCfiAddressable guard is added to skip elements that
foliate-js's CFI generator can't address (documentElement, body
itself, detached nodes, nodes outside <body>) — these previously
threw and spammed console.warn for every fragment, now they
silently fall back to the section CFI.
* perf(import): use native MOBI/AZW/AZW3 parser on Tauri targets
On Tauri (desktop/iOS/Android), importBook now forwards
MOBI/AZW/AZW3/PRC metadata + cover extraction to the Rust
parse_mobi_metadata command and reuses the partialMD5 it returns,
skipping the foliate-js full-buffer parse and the second pass over
the file for hashing. Mirrors the existing EPUB native fast-path
added in e3fc4767 — bookService tries EPUB first, then MOBI; both
bridges fall back to the foliate-js DocumentLoader when the native
path is unavailable (web target, parse error, format mismatch).
The new mobi_parser is built on the mobi crate (KF7+KF8 reader,
zero JS-side touch). It reads title, author, publisher, ISBN, ASIN,
publish date, language, subjects and description from the MobiHeader
+ EXTH records, resolves the EXTH 201 cover offset against the PDB
image-record table (with ThumbOffset / first-image fallbacks), and
strips KindleGen's HTML wrapping in EXTH 103 so the description goes
into the library DB as plain text. The parsed cover is funneled
through the same maybe_resize_cover path as EPUB, so MOBI library
thumbnails are also clamped to a 512px-long-edge JPEG.
Cover-resize / partialMD5 / RawCoverImage are extracted into a new
parser_common module shared between epub_parser and mobi_parser, so
a single tweak (e.g. raising the thumbnail target) applies to every
native importer and the partialMD5 implementation can't drift between
the two paths (a divergent algorithm would silently re-import every
existing book under a new hash on the first run).
Web targets and non-Kindle formats keep the existing path.
* test(tauri): verify native Rust EPUB parser parity with foliate-js
Add a Tauri WebView parity suite (epub-parser-parity.tauri.test.ts) that
cross-checks the native Rust parser against foliate-js on the same fixtures:
parse_epub_metadata / parse_epub_full (title, author, language, identifier,
publisher, published, subjects, partialMD5, OPF + per-entry size table), and
that opening with the native prefetch produces the same BookDoc and
computeBookNav (TOC) output as the pure-JS path.
Fix a parity divergence the suite caught: the Rust OPF parser mapped
dcterms:modified onto `published`, but foliate-js keeps them separate and
leaves `published` empty -- so EPUB3 books carrying only the mandatory
dcterms:modified got a bogus publication date on the native import path. Map
only dc:date now; add regression tests.
Test infra:
- vitest.tauri.config.mts: add optimizeDeps (mirroring vitest.browser.config)
so foliate-js-importing tauri tests load -- otherwise esbuild's dep scan
can't resolve '@pdfjs/pdf.min.mjs', pre-bundling is skipped, and the CJS
deps fail to import ("Importing a module script failed").
- capabilities-extra/webdriver.json: fix __test__ -> __tests__ fs scope typo
so import tests can open fixtures under src/__tests__/.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(import): foliate-js owns EPUB/MOBI metadata via standalone extractors
Rust contributes only the mechanical work that's expensive on a
WebView — partialMD5, the downscaled cover, and (for EPUB) the raw
OPF bytes Rust already had to read for cover resolution. Metadata
extraction is delegated to foliate-js's two new standalone entry
points (`parseEpubMetadataFromXML`, `readMobiMetadata`) so the
import-path BookDoc and the reader-path BookDoc share a single
parser implementation.
EPUB
- `parse_epub_metadata` returns
`{ partialMd5, cover, coverMime, opfPath, opfBytes }`. OPF bytes
are a free byproduct of the cover-resolution scan.
- `tryNativeParseEpub` runs `parseEpubMetadataFromXML` on the OPF
bytes and assembles a lightweight BookDoc stub (metadata +
getCover). The importer doesn't drive `DocumentLoader.open()`, so
no zip central-directory scan, no nav/ncx inflate, no spine walk.
- `coverMime` is preserved so `bookService.importBook`'s
`cover.type === 'image/svg+xml'` branch still routes SVG covers
through svg2png.
MOBI / AZW / AZW3 / PRC
- `parse_mobi_metadata` returns `{ partialMd5, cover, coverMime }`.
`tryNativeParseMobi` runs foliate's `readMobiMetadata` on the
same File, which uses `MOBI.open(file, { metadataOnly: true })`
to parse PalmDB + MobiHeader + EXTH and short-circuit before the
MOBI6 / KF8 init() that walks every text record.
- `Book.metadata.identifier` is foliate's `mobi.uid.toString()`
(PalmDB UID), the canonical MOBI identifier the reader path uses.
bookService.importBook
- EPUB and MOBI native branches consume the bridge's BookDoc stub
directly. The stub's `getCover()` returns the Rust-downscaled
blob, falling back to foliate's own `getCover` thunk when Rust
didn't extract a cover.
Other
- Drop the unused `base64` Rust dependency: cover bytes go over IPC
as `Vec<u8>` (Tauri 2 transports them natively, like opfBytes /
navBytes / ncxBytes).
- Drop the `nativePrefetch` option on `DocumentLoaderOptions`; no
caller passes it. `nativeFilePath` keeps driving `parse_epub_full`
on the open hot path.
Tests
- vitest.tauri parity test asserts byte-equal partialMD5, cover
presence parity, OPF bytes that decode to a real `<package>`
document, and that `parseEpubMetadataFromXML` on those bytes
produces the same user-visible metadata fields (title / author /
language / identifier / published) as `DocumentLoader.open()`.
* test(tauri): add War and Peace MOBI fixture for native parser parity
The .tauri parser-parity suite previously had no .mobi/.azw3 asset, so the native MOBI parser (metadata + EXTH cover resolution) was uncovered. Adds a real KF8 MOBI ("War and Peace") to enable MOBI parity coverage against foliate-js.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(foliate-js): bump submodule to readest/foliate-js main (91191ca)
Replaces the ad-hoc 02f435a with the merged main commit 91191ca, which lands the standalone OPF/MOBI metadata extractors (parseEpubMetadataFromXML, readMobiMetadata) the import fast-path depends on (foliate#19), plus the RTL multi-view rect-mapper fix (foliate#20). The extractor code is byte-identical to 02f435a, so the bridges are unaffected.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Huang Xin <chrox.huang@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
loveheaven
Generated
+152
-3
@@ -17,14 +17,19 @@ dependencies = [
|
||||
"discord-rich-presence",
|
||||
"futures",
|
||||
"futures-util",
|
||||
"image",
|
||||
"libc",
|
||||
"log",
|
||||
"md-5",
|
||||
"mobi",
|
||||
"objc",
|
||||
"objc-foundation",
|
||||
"objc2",
|
||||
"objc2-authentication-services",
|
||||
"objc2-foundation",
|
||||
"objc_id",
|
||||
"percent-encoding",
|
||||
"quick-xml 0.36.2",
|
||||
"rand 0.8.6",
|
||||
"read-progress-stream",
|
||||
"reqwest 0.12.28",
|
||||
@@ -62,6 +67,7 @@ dependencies = [
|
||||
"tokio",
|
||||
"tokio-util",
|
||||
"walkdir",
|
||||
"zip 2.4.2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -1133,6 +1139,12 @@ dependencies = [
|
||||
"objc",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "color_quant"
|
||||
version = "1.1.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "3d7b894f5411737b7867f4827955924d7c254fc9f4d91a6aad6b097804b1018b"
|
||||
|
||||
[[package]]
|
||||
name = "colorchoice"
|
||||
version = "1.0.5"
|
||||
@@ -1957,6 +1969,70 @@ version = "1.2.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "4ef6b89e5b37196644d8796de5268852ff179b44e96276cf4290264843743bb7"
|
||||
|
||||
[[package]]
|
||||
name = "encoding"
|
||||
version = "0.2.33"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "6b0d943856b990d12d3b55b359144ff341533e516d94098b1d3fc1ac666d36ec"
|
||||
dependencies = [
|
||||
"encoding-index-japanese",
|
||||
"encoding-index-korean",
|
||||
"encoding-index-simpchinese",
|
||||
"encoding-index-singlebyte",
|
||||
"encoding-index-tradchinese",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "encoding-index-japanese"
|
||||
version = "1.20141219.5"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "04e8b2ff42e9a05335dbf8b5c6f7567e5591d0d916ccef4e0b1710d32a0d0c91"
|
||||
dependencies = [
|
||||
"encoding_index_tests",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "encoding-index-korean"
|
||||
version = "1.20141219.5"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "4dc33fb8e6bcba213fe2f14275f0963fd16f0a02c878e3095ecfdf5bee529d81"
|
||||
dependencies = [
|
||||
"encoding_index_tests",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "encoding-index-simpchinese"
|
||||
version = "1.20141219.5"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "d87a7194909b9118fc707194baa434a4e3b0fb6a5a757c73c3adb07aa25031f7"
|
||||
dependencies = [
|
||||
"encoding_index_tests",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "encoding-index-singlebyte"
|
||||
version = "1.20141219.5"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "3351d5acffb224af9ca265f435b859c7c01537c0849754d3db3fdf2bfe2ae84a"
|
||||
dependencies = [
|
||||
"encoding_index_tests",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "encoding-index-tradchinese"
|
||||
version = "1.20141219.5"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "fd0e20d5688ce3cab59eb3ef3a2083a5c77bf496cb798dc6fcdb75f323890c18"
|
||||
dependencies = [
|
||||
"encoding_index_tests",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "encoding_index_tests"
|
||||
version = "0.1.4"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "a246d82be1c9d791c5dfde9a2bd045fc3cbba3fa2b11ad558f27d01712f00569"
|
||||
|
||||
[[package]]
|
||||
name = "encoding_rs"
|
||||
version = "0.8.35"
|
||||
@@ -2640,6 +2716,16 @@ dependencies = [
|
||||
"polyval",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "gif"
|
||||
version = "0.14.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "ee8cfcc411d9adbbaba82fb72661cc1bcca13e8bba98b364e62b2dba8f960159"
|
||||
dependencies = [
|
||||
"color_quant",
|
||||
"weezl",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "gio"
|
||||
version = "0.18.4"
|
||||
@@ -3293,10 +3379,14 @@ checksum = "85ab80394333c02fe689eaf900ab500fbd0c2213da414687ebf995a65d5a6104"
|
||||
dependencies = [
|
||||
"bytemuck",
|
||||
"byteorder-lite",
|
||||
"color_quant",
|
||||
"gif",
|
||||
"moxcms",
|
||||
"num-traits",
|
||||
"png 0.18.1",
|
||||
"tiff",
|
||||
"zune-core",
|
||||
"zune-jpeg",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -3955,6 +4045,16 @@ version = "0.8.4"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "47e1ffaa40ddd1f3ed91f717a33c8c0ee23fff369e3aa8772b9605cc1d22f4c3"
|
||||
|
||||
[[package]]
|
||||
name = "md-5"
|
||||
version = "0.10.6"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "d89e7ee0cfbedfc4da3340218492196241d89eefb6dab27de5df917a6d2e78cf"
|
||||
dependencies = [
|
||||
"cfg-if",
|
||||
"digest",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "measure_time"
|
||||
version = "0.9.0"
|
||||
@@ -4056,6 +4156,17 @@ dependencies = [
|
||||
"windows-sys 0.61.2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "mobi"
|
||||
version = "0.8.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "8f3f8e34216126be00a189105bda27462e1743d59cd4e15d6b99ff4af051b1b4"
|
||||
dependencies = [
|
||||
"encoding",
|
||||
"indexmap 1.9.3",
|
||||
"thiserror 1.0.69",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "moxcms"
|
||||
version = "0.8.1"
|
||||
@@ -5279,7 +5390,7 @@ checksum = "092791278e026273c1b65bbdcfbba3a300f2994c896bd01ab01da613c29c46f1"
|
||||
dependencies = [
|
||||
"base64 0.22.1",
|
||||
"indexmap 2.14.0",
|
||||
"quick-xml",
|
||||
"quick-xml 0.39.4",
|
||||
"serde",
|
||||
"time",
|
||||
]
|
||||
@@ -5562,6 +5673,15 @@ version = "2.0.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "a993555f31e5a609f617c12db6250dedcac1b0a85076912c436e6fc9b2c8e6a3"
|
||||
|
||||
[[package]]
|
||||
name = "quick-xml"
|
||||
version = "0.36.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "f7649a7b4df05aed9ea7ec6f628c67c9953a43869b8bc50929569b2999d443fe"
|
||||
dependencies = [
|
||||
"memchr",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "quick-xml"
|
||||
version = "0.39.4"
|
||||
@@ -7968,7 +8088,7 @@ dependencies = [
|
||||
"tokio",
|
||||
"url",
|
||||
"windows-sys 0.60.2",
|
||||
"zip",
|
||||
"zip 4.6.1",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -9435,7 +9555,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "9c324a910fd86ebdc364a3e61ec1f11737d3b1d6c273c0239ee8ff4bc0d24b4a"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quick-xml",
|
||||
"quick-xml 0.39.4",
|
||||
"quote",
|
||||
]
|
||||
|
||||
@@ -10724,6 +10844,23 @@ dependencies = [
|
||||
"syn 2.0.117",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "zip"
|
||||
version = "2.4.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "fabe6324e908f85a1c52063ce7aa26b68dcb7eb6dbc83a2d148403c9bc3eba50"
|
||||
dependencies = [
|
||||
"arbitrary",
|
||||
"crc32fast",
|
||||
"crossbeam-utils",
|
||||
"displaydoc",
|
||||
"flate2",
|
||||
"indexmap 2.14.0",
|
||||
"memchr",
|
||||
"thiserror 2.0.18",
|
||||
"zopfli",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "zip"
|
||||
version = "4.6.1"
|
||||
@@ -10742,6 +10879,18 @@ version = "1.0.21"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "b8848ee67ecc8aedbaf3e4122217aff892639231befc6a1b58d29fff4c2cabaa"
|
||||
|
||||
[[package]]
|
||||
name = "zopfli"
|
||||
version = "0.8.3"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "f05cd8797d63865425ff89b5c4a48804f35ba0ce8d125800027ad6017d2b5249"
|
||||
dependencies = [
|
||||
"bumpalo",
|
||||
"crc32fast",
|
||||
"log",
|
||||
"simd-adler32",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "zstd"
|
||||
version = "0.13.3"
|
||||
|
||||
@@ -64,6 +64,37 @@ tauri-plugin-device-info = "1.0.1"
|
||||
tauri-plugin-turso = { path = "./plugins/tauri-plugin-turso" }
|
||||
tauri-plugin-webdriver = { version = "0.2", optional = true }
|
||||
|
||||
# Native EPUB import path (Q1): zip + quick-xml + md5.
|
||||
# Used by `epub_parser::parse_epub_metadata` (partialMD5 + downscaled
|
||||
# cover) and `parse_epub_full` (OPF/nav/ncx prefetch + entry-size
|
||||
# table) to do the mechanical zip work without ferrying multi-MB blobs
|
||||
# across the JS<->Rust IPC boundary. Pure-Rust crates so they ship to
|
||||
# every Tauri target (desktop, iOS, Android) without extra system deps.
|
||||
zip = { version = "2", default-features = false, features = ["deflate"] }
|
||||
quick-xml = "0.36"
|
||||
md-5 = "0.10"
|
||||
# Used by `epub_parser::read_zip_entry` as a fallback when an OPF/manifest
|
||||
# href is percent-encoded (e.g. spaces as %20, CJK paths) but the zip stores
|
||||
# the raw decoded bytes — or vice versa. Already in our transitive dep graph
|
||||
# (reqwest pulls it), so adding it explicitly costs nothing.
|
||||
percent-encoding = "2"
|
||||
|
||||
# Cover thumbnail generation (Q2). We decode the cover image extracted
|
||||
# from the EPUB and, when its long edge exceeds the library-grid size,
|
||||
# re-encode a smaller JPEG so the on-disk `cover.png` is suitable for
|
||||
# webview rendering (~30-60 KB instead of multi-MB). Default features
|
||||
# are disabled to keep the binary lean — we only need decoders for the
|
||||
# formats EPUBs actually use (jpeg/png/gif) plus the JPEG encoder.
|
||||
image = { version = "0.25", default-features = false, features = ["jpeg", "png", "gif"] }
|
||||
|
||||
# Native MOBI/AZW/AZW3 import path. Mirrors the EPUB fast-path: parse
|
||||
# PalmDB + MobiHeader + EXTH in Rust to extract title/author/publisher/
|
||||
# description/isbn/asin/subjects/language and the cover image, then
|
||||
# return them to JS so the importer skips the foliate-js MOBI parser
|
||||
# (which has to inflate the whole record stream up-front on the iOS
|
||||
# WebView). Pure-Rust crate, ships to every Tauri target.
|
||||
mobi = "0.8"
|
||||
|
||||
[target."cfg(target_os = \"macos\")".dependencies]
|
||||
rand = "0.8"
|
||||
cocoa = "0.25"
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
"urls": ["http://127.0.0.1:*", "http://localhost:*"]
|
||||
},
|
||||
"local": false,
|
||||
"windows": ["main"],
|
||||
"windows": ["main", "reader-*"],
|
||||
"permissions": [
|
||||
"core:default",
|
||||
"fs:default",
|
||||
@@ -18,6 +18,8 @@
|
||||
"allow": [
|
||||
{ "path": "**/.readest-test-sandbox-tauri" },
|
||||
{ "path": "**/.readest-test-sandbox-tauri/**" },
|
||||
{ "path": "**/__tests__" },
|
||||
{ "path": "**/__tests__/**" },
|
||||
{ "path": "**/Readest" },
|
||||
{ "path": "**/Readest/**" }
|
||||
]
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -26,8 +26,11 @@ mod clip_url;
|
||||
mod dir_scanner;
|
||||
#[cfg(any(target_os = "macos", target_os = "windows", target_os = "linux"))]
|
||||
mod discord_rpc;
|
||||
mod epub_parser;
|
||||
#[cfg(target_os = "macos")]
|
||||
mod macos;
|
||||
mod mobi_parser;
|
||||
mod parser_common;
|
||||
mod transfer_file;
|
||||
#[cfg(desktop)]
|
||||
mod window_state;
|
||||
@@ -268,6 +271,11 @@ pub fn run() {
|
||||
get_executable_dir,
|
||||
allow_paths_in_scopes,
|
||||
dir_scanner::read_dir,
|
||||
epub_parser::parse_epub_metadata,
|
||||
epub_parser::extract_epub_cover_full,
|
||||
epub_parser::parse_epub_full,
|
||||
mobi_parser::parse_mobi_metadata,
|
||||
mobi_parser::extract_mobi_cover_full,
|
||||
#[cfg(target_os = "macos")]
|
||||
macos::safari_auth::auth_with_safari,
|
||||
#[cfg(target_os = "macos")]
|
||||
|
||||
@@ -0,0 +1,246 @@
|
||||
// Native MOBI/AZW/AZW3 import path.
|
||||
//
|
||||
// Scope after PR review (mirrors `epub_parser`): this command no
|
||||
// longer extracts MOBI metadata. The full set of EXTH semantics
|
||||
// (placeholder filtering, ISBN-10/13 checksum validation, ASIN
|
||||
// fallback, BCP-47 language normalisation, " & " / " and " / ";"
|
||||
// author splitting, HTML-stripped descriptions, …) is left to
|
||||
// foliate-js's `mobi.js`, which the JS side already runs through
|
||||
// `DocumentLoader.open()` on the import path. Re-implementing those
|
||||
// rules in Rust would silently diverge from the canonical parser
|
||||
// used by the reader hot path; worse, switching `Book.metadata.identifier`
|
||||
// from the foliate `mobi.uid` (PalmDB UID) to a Rust-derived ISBN/ASIN
|
||||
// would change the metaHash for every existing MOBI in users' libraries
|
||||
// and trigger spurious re-imports on upgrade.
|
||||
//
|
||||
// What `parse_mobi_metadata` still does on the import hot path:
|
||||
// - compute partialMD5 over the file (matches `utils/md5.ts::partialMD5`,
|
||||
// shared with `epub_parser` via `parser_common::compute_partial_md5`);
|
||||
// - parse the PalmDB / MobiHeader / EXTH headers via the `mobi` crate
|
||||
// just enough to locate the cover record;
|
||||
// - locate the cover image (EXTH `CoverOffset` 201 → `ThumbOffset` 202
|
||||
// → first image record fallback), sniff the MIME from magic bytes,
|
||||
// and run `parser_common::maybe_resize_cover` to clamp it to the
|
||||
// library-grid thumbnail target. Cover decode/resize stays here
|
||||
// because the `image` crate is materially faster than the
|
||||
// `createImageBitmap` + canvas round-trip on Android mid-tier
|
||||
// devices, and bulk imports actually exercise that.
|
||||
//
|
||||
// Returned to JS via `parse_mobi_metadata`. The JS bridge wraps the
|
||||
// downscaled bytes into a Blob and Proxies it onto foliate's BookDoc
|
||||
// in place of the foliate `getCover()` (which would otherwise return
|
||||
// the original-resolution bytes). foliate stays the source of truth
|
||||
// for everything else.
|
||||
|
||||
use mobi::headers::ExthRecord;
|
||||
use mobi::Mobi;
|
||||
use serde::Serialize;
|
||||
use std::path::Path;
|
||||
|
||||
use crate::parser_common::{compute_partial_md5, maybe_resize_cover, RawCoverImage};
|
||||
|
||||
#[derive(Debug, Serialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct ParsedMobi {
|
||||
pub partial_md5: String,
|
||||
/// `None` when the MOBI has no embedded cover image. Otherwise
|
||||
/// the pre-resized bytes + sniffed MIME — the JS side wraps them
|
||||
/// in a Blob and overrides foliate's `getCover()` so the on-disk
|
||||
/// thumbnail wins over foliate's full-resolution decode.
|
||||
pub cover: Option<RawCoverImage>,
|
||||
}
|
||||
|
||||
/// Tauri command: parse a MOBI/AZW/AZW3 file's partialMD5 + cover and
|
||||
/// return both in one IPC round-trip.
|
||||
///
|
||||
/// Runs on a blocking pool because `mobi::Mobi::from_path` reads the
|
||||
/// whole file synchronously and parsing a 50 MB AZW3 can take tens of
|
||||
/// milliseconds — long enough to want it off the Tauri main runtime.
|
||||
#[tauri::command]
|
||||
pub async fn parse_mobi_metadata(file_path: String) -> Result<ParsedMobi, String> {
|
||||
tauri::async_runtime::spawn_blocking(move || parse_mobi_metadata_sync(&file_path))
|
||||
.await
|
||||
.map_err(|e| format!("join error: {e}"))?
|
||||
}
|
||||
|
||||
fn parse_mobi_metadata_sync(file_path: &str) -> Result<ParsedMobi, String> {
|
||||
let path = Path::new(file_path);
|
||||
if !path.is_file() {
|
||||
return Err(format!("file not found: {file_path}"));
|
||||
}
|
||||
|
||||
let partial_md5 = compute_partial_md5(path).map_err(|e| format!("partial_md5 failed: {e}"))?;
|
||||
|
||||
let mobi = Mobi::from_path(path).map_err(|e| format!("parse mobi: {e}"))?;
|
||||
|
||||
let cover = extract_cover(&mobi).map(|raw| {
|
||||
let (bytes, mime) = maybe_resize_cover(raw.bytes, &raw.mime);
|
||||
RawCoverImage { bytes, mime }
|
||||
});
|
||||
|
||||
Ok(ParsedMobi { partial_md5, cover })
|
||||
}
|
||||
|
||||
/// Extract the *original* (un-resized) cover bytes from a MOBI / AZW / AZW3.
|
||||
///
|
||||
/// Mirrors `epub_parser::extract_epub_cover_full`: the import path stores a
|
||||
/// downscaled thumbnail (via `maybe_resize_cover` inside
|
||||
/// `parse_mobi_metadata_sync`) for the library grid, but features like the
|
||||
/// Android / iOS lock-screen wallpaper want the full-resolution artwork. We
|
||||
/// re-run the same EXTH lookup as the import path here, but skip the
|
||||
/// downscale step and hand the raw record bytes back to JS along with a
|
||||
/// MIME sniffed from the magic bytes.
|
||||
///
|
||||
/// Returns `Err` only when the file has no embedded cover at all.
|
||||
#[tauri::command]
|
||||
pub async fn extract_mobi_cover_full(file_path: String) -> Result<RawCoverImage, String> {
|
||||
tauri::async_runtime::spawn_blocking(move || extract_mobi_cover_full_sync(&file_path))
|
||||
.await
|
||||
.map_err(|e| format!("join error: {e}"))?
|
||||
}
|
||||
|
||||
fn extract_mobi_cover_full_sync(file_path: &str) -> Result<RawCoverImage, String> {
|
||||
let path = Path::new(file_path);
|
||||
if !path.is_file() {
|
||||
return Err(format!("file not found: {file_path}"));
|
||||
}
|
||||
let mobi = Mobi::from_path(path).map_err(|e| format!("parse mobi: {e}"))?;
|
||||
extract_cover(&mobi).ok_or_else(|| "no cover image in mobi".to_string())
|
||||
}
|
||||
|
||||
/// Locate and decode the embedded cover image.
|
||||
///
|
||||
/// Strategy (mirrors what foliate-js's mobi.js does on the JS side):
|
||||
/// 1. Read EXTH `CoverOffset` (record 201). The payload is a 4-byte
|
||||
/// big-endian u32 giving an offset into the image-record subset.
|
||||
/// Add `MobiHeader.first_image_index` to get a global PDB record
|
||||
/// index, then look that record up in `Mobi::image_records()`.
|
||||
/// 2. If 201 is missing, try `ThumbOffset` (record 202) the same way.
|
||||
/// 3. If neither is present, fall back to the first image record —
|
||||
/// MOBI generators almost always place the cover first, and a
|
||||
/// "wrong but plausible" thumbnail is better than no thumbnail.
|
||||
///
|
||||
/// Returns `None` only when the file has no image records at all (rare
|
||||
/// for real Kindle content).
|
||||
fn extract_cover(mobi: &Mobi) -> Option<RawCoverImage> {
|
||||
let images = mobi.image_records();
|
||||
if images.is_empty() {
|
||||
return None;
|
||||
}
|
||||
|
||||
let first_image_index = mobi.metadata.mobi.first_image_index;
|
||||
|
||||
let exth_offset = read_exth_u32(mobi, ExthRecord::CoverOffset)
|
||||
.or_else(|| read_exth_u32(mobi, ExthRecord::ThumbOffset));
|
||||
|
||||
let bytes: Vec<u8> = if let Some(off) = exth_offset {
|
||||
// EXTH stores a *relative* offset; the absolute PDB record id
|
||||
// is `first_image_index + off`. `image_records()` is filtered
|
||||
// to image-only records, so we have to find the entry whose
|
||||
// PdbRecord id matches the absolute id, not index linearly.
|
||||
let target_id = first_image_index.saturating_add(off);
|
||||
images
|
||||
.iter()
|
||||
.find(|r| r.record.id == target_id)
|
||||
// Some files store the offset already pre-resolved into
|
||||
// image_records()'s ordering; allow that as a fallback.
|
||||
.or_else(|| images.get(off as usize))
|
||||
.map(|r| r.content.to_vec())
|
||||
.unwrap_or_else(|| images[0].content.to_vec())
|
||||
} else {
|
||||
images[0].content.to_vec()
|
||||
};
|
||||
|
||||
if bytes.is_empty() {
|
||||
return None;
|
||||
}
|
||||
|
||||
let mime = sniff_image_mime(&bytes).to_string();
|
||||
Some(RawCoverImage { bytes, mime })
|
||||
}
|
||||
|
||||
/// Read the first occurrence of `record` and interpret its payload as
|
||||
/// a 4-byte big-endian u32. EXTH offset records (201 / 202 / 116, etc.)
|
||||
/// follow this convention. Returns `None` if the record is absent or
|
||||
/// shorter than 4 bytes.
|
||||
fn read_exth_u32(mobi: &Mobi, record: ExthRecord) -> Option<u32> {
|
||||
let recs = mobi.metadata.exth.get_record(record)?;
|
||||
let bytes = recs.first()?;
|
||||
if bytes.len() < 4 {
|
||||
return None;
|
||||
}
|
||||
Some(u32::from_be_bytes([bytes[0], bytes[1], bytes[2], bytes[3]]))
|
||||
}
|
||||
|
||||
/// Best-effort MIME sniffing from magic bytes for the formats MOBI
|
||||
/// covers are realistically stored as. Falls back to "image/jpeg" —
|
||||
/// the dominant case — when the magic is unknown, because
|
||||
/// `image::load_from_memory` (called downstream by
|
||||
/// `maybe_resize_cover`) will detect the real format anyway and the
|
||||
/// hint MIME is only used when we *don't* re-encode (small covers,
|
||||
/// kept verbatim).
|
||||
///
|
||||
/// BMP is included because some early KindleGen builds (and a few
|
||||
/// self-published .prc files) shipped BMP covers; the JS thumbnail
|
||||
/// pipeline can render BMP via the same downscale path.
|
||||
fn sniff_image_mime(bytes: &[u8]) -> &'static str {
|
||||
if bytes.starts_with(&[0xFF, 0xD8, 0xFF]) {
|
||||
"image/jpeg"
|
||||
} else if bytes.starts_with(&[0x89, b'P', b'N', b'G', 0x0D, 0x0A, 0x1A, 0x0A]) {
|
||||
"image/png"
|
||||
} else if bytes.starts_with(b"GIF87a") || bytes.starts_with(b"GIF89a") {
|
||||
"image/gif"
|
||||
} else if bytes.starts_with(b"RIFF") && bytes.get(8..12) == Some(b"WEBP") {
|
||||
"image/webp"
|
||||
} else if bytes.starts_with(b"BM") {
|
||||
"image/bmp"
|
||||
} else {
|
||||
"image/jpeg"
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn sniff_image_mime_jpeg() {
|
||||
assert_eq!(sniff_image_mime(&[0xFF, 0xD8, 0xFF, 0xE0]), "image/jpeg");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn sniff_image_mime_png() {
|
||||
assert_eq!(
|
||||
sniff_image_mime(&[0x89, b'P', b'N', b'G', 0x0D, 0x0A, 0x1A, 0x0A, 0, 0]),
|
||||
"image/png"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn sniff_image_mime_gif() {
|
||||
assert_eq!(sniff_image_mime(b"GIF89a..."), "image/gif");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn sniff_image_mime_webp() {
|
||||
let mut buf = Vec::new();
|
||||
buf.extend_from_slice(b"RIFF");
|
||||
buf.extend_from_slice(&[0, 0, 0, 0]);
|
||||
buf.extend_from_slice(b"WEBP");
|
||||
assert_eq!(sniff_image_mime(&buf), "image/webp");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn sniff_image_mime_unknown_falls_back_to_jpeg() {
|
||||
assert_eq!(sniff_image_mime(&[0, 0, 0, 0]), "image/jpeg");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn sniff_image_mime_bmp() {
|
||||
// BMP magic is "BM" followed by file size + reserved + offset.
|
||||
let mut buf = Vec::new();
|
||||
buf.extend_from_slice(b"BM");
|
||||
buf.extend_from_slice(&[0; 12]);
|
||||
assert_eq!(sniff_image_mime(&buf), "image/bmp");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,146 @@
|
||||
// Shared helpers for the native import fast-path.
|
||||
//
|
||||
// Both the EPUB parser (`epub_parser`) and the MOBI/AZW/AZW3 parser
|
||||
// (`mobi_parser`) need to:
|
||||
// - compute the same `partialMD5` over the input file as `utils/md5.ts`,
|
||||
// so the on-disk `Books/<hash>/...` layout stays stable regardless of
|
||||
// which parser produced the entry,
|
||||
// - clamp oversized cover artwork to the library-grid thumbnail size,
|
||||
// re-encoding as JPEG q85 when downscaling actually fires.
|
||||
//
|
||||
// Keeping these in a single module avoids drift between the two import
|
||||
// paths (a divergent partialMD5 implementation would silently re-import
|
||||
// every existing book under a new hash on the first run after a change).
|
||||
//
|
||||
// `RawCoverImage` is the IPC-shaped struct returned to JS as a byte array
|
||||
// + MIME pair; the JS bridges (`tauriEpubBridge.ts`, `tauriMobiBridge.ts`)
|
||||
// turn it back into a `Uint8Array` before persisting through the existing
|
||||
// `Books/<hash>/cover.<ext>` path.
|
||||
|
||||
use image::{codecs::jpeg::JpegEncoder, imageops::FilterType, GenericImageView};
|
||||
use md5::{Digest, Md5};
|
||||
use serde::Serialize;
|
||||
use std::fs::File;
|
||||
use std::io::{Cursor, Read, Seek, SeekFrom};
|
||||
use std::path::Path;
|
||||
|
||||
/// Cover thumbnail target. Sized for the library grid (~250-300px @2x)
|
||||
/// and the reader-sidebar / detail-view rows (which are smaller still).
|
||||
/// Anything whose long edge is already at or below this stays untouched —
|
||||
/// no decode/re-encode, original bytes are kept verbatim. Anything larger
|
||||
/// is downscaled with [`COVER_RESIZE_FILTER`] and re-encoded as JPEG q85.
|
||||
pub const COVER_MAX_LONG_EDGE: u32 = 512;
|
||||
pub const COVER_JPEG_QUALITY: u8 = 85;
|
||||
|
||||
/// Resampling filter used to downscale covers. We deliberately use
|
||||
/// `Triangle` (4-tap bilinear-ish) instead of `Lanczos3` (36-tap): at the
|
||||
/// 512px-thumbnail scale the visual difference is imperceptible, but
|
||||
/// Triangle is ~5-8x faster on a debug build (and ~3-5x faster on release)
|
||||
/// because it touches far fewer source pixels per output pixel. Cover
|
||||
/// thumbnails are displayed at <=300px in the UI, so any sharpening
|
||||
/// advantage Lanczos3 would have is moot.
|
||||
pub const COVER_RESIZE_FILTER: FilterType = FilterType::Triangle;
|
||||
|
||||
#[derive(Debug, Serialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct RawCoverImage {
|
||||
/// Raw image bytes (serde will encode this as a JS array; the JS side
|
||||
/// converts it back to a Uint8Array before writing to disk).
|
||||
pub bytes: Vec<u8>,
|
||||
pub mime: String,
|
||||
}
|
||||
|
||||
/// Decode `bytes`, and if the long edge exceeds [`COVER_MAX_LONG_EDGE`],
|
||||
/// resize ([`COVER_RESIZE_FILTER`], aspect ratio preserved) and re-encode
|
||||
/// as JPEG at [`COVER_JPEG_QUALITY`].
|
||||
///
|
||||
/// On any decode/encode failure we fall back to the original bytes + the
|
||||
/// caller-provided MIME so a malformed (but viewable) cover still makes it
|
||||
/// to disk. `hint_mime` is informative only — `image::load_from_memory`
|
||||
/// sniffs the actual format from the magic bytes, so misclaimed MIMEs in
|
||||
/// the source container don't trip us up.
|
||||
pub fn maybe_resize_cover(bytes: Vec<u8>, hint_mime: &str) -> (Vec<u8>, String) {
|
||||
let img = match image::load_from_memory(&bytes) {
|
||||
Ok(i) => i,
|
||||
Err(_) => return (bytes, hint_mime.to_string()),
|
||||
};
|
||||
let (w, h) = img.dimensions();
|
||||
if w.max(h) <= COVER_MAX_LONG_EDGE {
|
||||
return (bytes, hint_mime.to_string());
|
||||
}
|
||||
let resized = img.resize(
|
||||
COVER_MAX_LONG_EDGE,
|
||||
COVER_MAX_LONG_EDGE,
|
||||
COVER_RESIZE_FILTER,
|
||||
);
|
||||
let rgb = resized.to_rgb8();
|
||||
|
||||
let mut out = Vec::with_capacity(64 * 1024);
|
||||
{
|
||||
let mut encoder = JpegEncoder::new_with_quality(Cursor::new(&mut out), COVER_JPEG_QUALITY);
|
||||
if encoder
|
||||
.encode(
|
||||
rgb.as_raw(),
|
||||
rgb.width(),
|
||||
rgb.height(),
|
||||
image::ExtendedColorType::Rgb8,
|
||||
)
|
||||
.is_err()
|
||||
{
|
||||
return (bytes, hint_mime.to_string());
|
||||
}
|
||||
}
|
||||
(out, "image/jpeg".to_string())
|
||||
}
|
||||
|
||||
/// Mirror of `utils/md5.ts::partialMD5`:
|
||||
/// step = 1024, size = 1024
|
||||
/// for i in -1..=10:
|
||||
/// start = min(file.size, step << (2*i)) // JS 32-bit shift
|
||||
/// end = min(start + size, file.size)
|
||||
/// if start >= file.size: break
|
||||
/// hash file[start..end]
|
||||
///
|
||||
/// JS bit-shift operands are masked to their low 5 bits, so `1024 << -2`
|
||||
/// actually means `1024 << 30`, which is far larger than any reasonable
|
||||
/// file. That makes the very first iteration (i = -1) immediately break
|
||||
/// for files smaller than ~1 GiB, leaving the hasher empty -> md5 of "" =
|
||||
/// d41d8cd9... We must reproduce that behaviour bit-for-bit so existing
|
||||
/// on-disk hashes (Books/<hash>/...) keep matching.
|
||||
pub fn compute_partial_md5(path: &Path) -> std::io::Result<String> {
|
||||
const STEP: u32 = 1024;
|
||||
const CHUNK: u64 = 1024;
|
||||
|
||||
let mut file = File::open(path)?;
|
||||
let file_len = file.metadata()?.len();
|
||||
|
||||
let mut hasher = Md5::new();
|
||||
let mut buf = vec![0u8; CHUNK as usize];
|
||||
|
||||
for i in -1i32..=10 {
|
||||
// JS evaluates `step << (2*i)` as a 32-bit shift, where the operand is
|
||||
// implicitly masked to its low 5 bits. So `1024 << -2` is the same as
|
||||
// `1024 << 30`, which overflows i32 to 0 (the high bits are dropped).
|
||||
// For i = 0..=4 the shift is 0..=8 and stays within i32; for i >= 5
|
||||
// the result overflows to 0 again. We mirror that with wrapping_shl.
|
||||
let shift_amount = ((2 * i) as u32) & 31;
|
||||
let shifted = (STEP as i32).wrapping_shl(shift_amount);
|
||||
// Negative i32 results coerce to 0 here. JS's Math.min would surface
|
||||
// the negative value, but the subsequent `start >= file.size` check
|
||||
// would skip the read; clamping to 0 gives the same observable
|
||||
// hash for non-empty files while avoiding negative seek offsets.
|
||||
let raw = shifted.max(0) as u64;
|
||||
let start = std::cmp::min(file_len, raw);
|
||||
if start >= file_len {
|
||||
break;
|
||||
}
|
||||
let end = std::cmp::min(start + CHUNK, file_len);
|
||||
let to_read = (end - start) as usize;
|
||||
file.seek(SeekFrom::Start(start))?;
|
||||
let slice = &mut buf[..to_read];
|
||||
file.read_exact(slice)?;
|
||||
hasher.update(&slice[..]);
|
||||
}
|
||||
|
||||
Ok(format!("{:x}", hasher.finalize()))
|
||||
}
|
||||
Binary file not shown.
@@ -0,0 +1,249 @@
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import { invoke } from './tauri-invoke';
|
||||
import { DocumentLoader } from '@/libs/document';
|
||||
import type { BookDoc, TOCItem, SectionItem } from '@/libs/document';
|
||||
import { computeBookNav, type BookNav } from '@/services/nav';
|
||||
import { partialMD5 } from '@/utils/md5';
|
||||
import { formatAuthors, formatTitle, getPrimaryLanguage } from '@/utils/book';
|
||||
|
||||
/**
|
||||
* Cross-language parity tests for the native Rust EPUB parser (PR #4369).
|
||||
*
|
||||
* These run inside the real Tauri WebView (see scripts/test-tauri.sh), which
|
||||
* is the only environment where both parsers are reachable at once:
|
||||
* - the Rust commands (`parse_epub_metadata` / `parse_epub_full`) via the
|
||||
* Tauri IPC `invoke()`, and
|
||||
* - the foliate-js parser via `DocumentLoader`, running in the WebView's JS.
|
||||
*
|
||||
* The Rust commands read by absolute on-disk path; `process.env.CWD` (injected
|
||||
* by vitest.tauri.config.mts) gives us the readest-app dir so we can build that
|
||||
* path. The JS side fetches the *same* file through a Vite-served URL.
|
||||
*
|
||||
* MOBI/AZW parity is intentionally not covered here: there is no Kindle-format
|
||||
* fixture in the repo to feed both parsers, and the Rust MOBI path is exercised
|
||||
* by `mobi_parser`'s own unit tests. Add a `.mobi` fixture to extend this.
|
||||
*/
|
||||
|
||||
// CWD is the absolute readest-app directory (process.cwd() at config load).
|
||||
const CWD = process.env['CWD'] as string;
|
||||
|
||||
const EPUB_FIXTURES = [
|
||||
'sample-alice.epub', // NCX TOC, full metadata (author/publisher/date/subjects/cover)
|
||||
'repro-3688.epub', // NCX TOC, fragment-anchored TOC hrefs (#ch01), plain-string author
|
||||
'repro-3683.epub', // EPUB3 nav doc, dcterms:modified but no dc:date, no author
|
||||
] as const;
|
||||
|
||||
const diskPath = (name: string) => `${CWD}/src/__tests__/fixtures/data/${name}`;
|
||||
const fixtureUrl = (name: string) => new URL(`../fixtures/data/${name}`, import.meta.url).href;
|
||||
|
||||
// ─── Rust IPC return shapes (serde camelCase) ────────────────────────
|
||||
//
|
||||
// `parse_epub_metadata` deliberately does NOT carry OPF metadata —
|
||||
// foliate-js owns that on both platforms. Rust contributes only the
|
||||
// partialMD5 hash and the pre-resized cover bytes.
|
||||
interface RustParsedEpubMetadata {
|
||||
partialMd5: string;
|
||||
cover?: number[] | Uint8Array | null;
|
||||
coverMime?: string | null;
|
||||
opfPath: string;
|
||||
opfBytes: number[] | Uint8Array;
|
||||
}
|
||||
interface RustParsedEpubFull {
|
||||
partialMd5: string;
|
||||
opfPath: string;
|
||||
opfBytes: number[] | Uint8Array;
|
||||
navPath?: string | null;
|
||||
ncxPath?: string | null;
|
||||
sizes: Record<string, number>;
|
||||
}
|
||||
|
||||
// ─── helpers ─────────────────────────────────────────────────────────
|
||||
const fetchBytes = async (name: string): Promise<ArrayBuffer> =>
|
||||
(await fetch(fixtureUrl(name))).arrayBuffer();
|
||||
|
||||
const makeFile = (buf: ArrayBuffer, name: string): File =>
|
||||
new File([buf], name, { type: 'application/epub+zip' });
|
||||
|
||||
const openEpub = async (file: File, nativeFilePath?: string): Promise<BookDoc> => {
|
||||
const loader = new DocumentLoader(file, nativeFilePath ? { nativeFilePath } : {});
|
||||
return (await loader.open()).book;
|
||||
};
|
||||
|
||||
/**
|
||||
* User-visible author string, used by Block 3 (foliate-vs-foliate parity)
|
||||
* to confirm the native prefetch doesn't perturb metadata extraction.
|
||||
*/
|
||||
const jsAuthor = (book: BookDoc): string => {
|
||||
const a = book.metadata.author;
|
||||
return a == null ? '' : formatAuthors(a, book.metadata.language);
|
||||
};
|
||||
|
||||
const tocBrief = (items: TOCItem[] | undefined): unknown =>
|
||||
(items ?? []).map((i) => ({
|
||||
label: i.label,
|
||||
href: i.href,
|
||||
subitems: i.subitems?.length ? tocBrief(i.subitems) : undefined,
|
||||
}));
|
||||
|
||||
// id/size/linear only: foliate leaves SectionItem.href undefined, so it is not
|
||||
// a parity signal; the section identity + computed byte size + linear flag are.
|
||||
const sectionBrief = (sections: SectionItem[]) =>
|
||||
sections.map((s) => ({ id: s.id, size: s.size, linear: s.linear }));
|
||||
|
||||
const navFragmentMap = (
|
||||
nav: BookNav,
|
||||
): Record<string, Array<{ href: string; cfi: string; size: number }>> =>
|
||||
Object.fromEntries(
|
||||
Object.entries(nav.sections).map(([id, sec]) => [
|
||||
id,
|
||||
sec.fragments.map((f) => ({ href: f.href, cfi: f.cfi, size: f.size })),
|
||||
]),
|
||||
);
|
||||
|
||||
// ─── 1. Import-path: Rust contributes partialMD5 + cover + OPF bytes ─────
|
||||
//
|
||||
// `parse_epub_metadata` deliberately does not extract OPF metadata —
|
||||
// foliate-js's `parseEpubMetadataFromXML` does that on the JS side,
|
||||
// against the very `opfBytes` Rust hands over. The invariants we
|
||||
// assert here are the four contributions Rust still makes:
|
||||
// 1. partialMD5 byte-equal to the JS reference (the on-disk
|
||||
// Books/<hash>/ layout depends on byte-exact parity);
|
||||
// 2. cover presence matches what foliate's `EPUB.getCover()` would
|
||||
// surface (Rust downscales/re-encodes, so bytes differ by
|
||||
// design — only presence is a parity signal);
|
||||
// 3. `opfPath` resolves to a real OPF document — `opfBytes`
|
||||
// decode to a `<package>`-rooted XML;
|
||||
// 4. running foliate-js's exported `parseEpubMetadataFromXML` on
|
||||
// those bytes produces the same user-visible metadata fields
|
||||
// (title / author / language / identifier / published) as
|
||||
// driving `DocumentLoader.open()` against the same File. This
|
||||
// is the parity that protects the import-path BookDoc against
|
||||
// drift from the reader-path BookDoc, since the importer
|
||||
// consumes only the foliate-derived metadata + the Rust cover.
|
||||
describe('parse_epub_metadata: partialMD5 + cover + OPF parity', () => {
|
||||
for (const name of EPUB_FIXTURES) {
|
||||
it(`Rust mechanical work + JS-side OPF metadata parity: ${name}`, async () => {
|
||||
const buf = await fetchBytes(name);
|
||||
const file = makeFile(buf, name);
|
||||
|
||||
const rust = (await invoke('parse_epub_metadata', {
|
||||
filePath: diskPath(name),
|
||||
})) as RustParsedEpubMetadata;
|
||||
const js = await openEpub(file);
|
||||
|
||||
// 1. partialMD5 byte-equal to the JS reference.
|
||||
expect(rust.partialMd5).toBe(await partialMD5(file));
|
||||
|
||||
// 2. Cover presence parity.
|
||||
const jsHasCover = (await js.getCover()) != null;
|
||||
const rustHasCover =
|
||||
rust.cover != null &&
|
||||
(rust.cover instanceof Uint8Array ? rust.cover.byteLength : rust.cover.length) > 0;
|
||||
expect(rustHasCover).toBe(jsHasCover);
|
||||
|
||||
// 3. OPF bytes decode to a real package document.
|
||||
expect(rust.opfPath).toBeTruthy();
|
||||
const opfXml = new TextDecoder('utf-8').decode(
|
||||
rust.opfBytes instanceof Uint8Array ? rust.opfBytes : new Uint8Array(rust.opfBytes),
|
||||
);
|
||||
expect(opfXml).toContain('<package');
|
||||
|
||||
// 4. Running foliate-js's standalone OPF metadata extractor on
|
||||
// those bytes yields the same user-visible metadata fields
|
||||
// as the full DocumentLoader path. This is what the importer
|
||||
// actually consumes via `tryNativeParseEpub`.
|
||||
const epubModule = (await import('foliate-js/epub.js')) as unknown as {
|
||||
parseEpubMetadataFromXML: (xml: string) => { metadata: BookDoc['metadata'] };
|
||||
};
|
||||
const standalone = epubModule.parseEpubMetadataFromXML(opfXml);
|
||||
const fields = (m: BookDoc['metadata']) => ({
|
||||
title: formatTitle(m.title),
|
||||
author: m.author == null ? '' : formatAuthors(m.author, m.language),
|
||||
language: getPrimaryLanguage(m.language),
|
||||
identifier: m.identifier ?? null,
|
||||
published: m.published ?? '',
|
||||
});
|
||||
expect(fields(standalone.metadata)).toEqual(fields(js.metadata));
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
// ─── 2. Open-path prefetch parity (parse_epub_full size table + md5) ──────
|
||||
describe('parse_epub_full parity with the foliate-js zip loader', () => {
|
||||
for (const name of EPUB_FIXTURES) {
|
||||
it(`returns a coherent OPF + size table matching foliate-js: ${name}`, async () => {
|
||||
const buf = await fetchBytes(name);
|
||||
const file = makeFile(buf, name);
|
||||
|
||||
const full = (await invoke('parse_epub_full', {
|
||||
filePath: diskPath(name),
|
||||
})) as RustParsedEpubFull;
|
||||
const js = await openEpub(file);
|
||||
|
||||
// Same hash from both Rust commands and from JS.
|
||||
expect(full.partialMd5).toBe(await partialMD5(file));
|
||||
|
||||
// OPF bytes decode to a real package document.
|
||||
const opfXml = new TextDecoder('utf-8').decode(
|
||||
full.opfBytes instanceof Uint8Array ? full.opfBytes : new Uint8Array(full.opfBytes),
|
||||
);
|
||||
expect(opfXml).toContain('<package');
|
||||
|
||||
// Exactly one TOC source, matching the fixture's TOC kind.
|
||||
expect(Boolean(full.navPath) || Boolean(full.ncxPath)).toBe(true);
|
||||
|
||||
// The size table must cover every spine section foliate exposes, and the
|
||||
// uncompressed sizes must agree (foliate computes getSize from the same
|
||||
// zip central directory when the prefetch is absent).
|
||||
for (const section of js.sections) {
|
||||
expect(full.sizes[section.id]).toBe(section.size);
|
||||
}
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
// ─── 3. Behavioral parity: native prefetch vs pure foliate-js, incl. TOC ──
|
||||
describe('book open + TOC enrichment parity (native prefetch vs foliate-js)', () => {
|
||||
for (const name of EPUB_FIXTURES) {
|
||||
it(`produces an identical BookDoc and nav with vs without the Rust path: ${name}`, async () => {
|
||||
const buf = await fetchBytes(name);
|
||||
|
||||
// Prove the native prefetch is actually exercised (not silently falling
|
||||
// back), independently of DocumentLoader internals.
|
||||
const { tryNativePrefetchEpub } = await import('@/utils/tauriEpubBridge');
|
||||
const prefetch = await tryNativePrefetchEpub(diskPath(name));
|
||||
expect(prefetch).not.toBeNull();
|
||||
expect(prefetch!.textCache.has('META-INF/container.xml')).toBe(true);
|
||||
expect(prefetch!.partialMd5).toBe(await partialMD5(makeFile(buf, name)));
|
||||
|
||||
// Open the same file both ways. Separate File objects so the two zip
|
||||
// loaders don't share any state.
|
||||
const jsBook = await openEpub(makeFile(buf, name));
|
||||
const nativeBook = await openEpub(makeFile(buf, name), diskPath(name));
|
||||
|
||||
// Metadata that flows into the library DB must be identical.
|
||||
const pick = (b: BookDoc) => ({
|
||||
title: formatTitle(b.metadata.title),
|
||||
author: jsAuthor(b),
|
||||
language: getPrimaryLanguage(b.metadata.language),
|
||||
identifier: b.metadata.identifier ?? null,
|
||||
published: b.metadata.published ?? '',
|
||||
});
|
||||
expect(pick(nativeBook)).toEqual(pick(jsBook));
|
||||
|
||||
// Spine + TOC structure must be identical.
|
||||
expect(sectionBrief(nativeBook.sections)).toEqual(sectionBrief(jsBook.sections));
|
||||
expect(tocBrief(nativeBook.toc)).toEqual(tocBrief(jsBook.toc));
|
||||
|
||||
// computeBookNav runs the parallelized section scan, fragment-CFI math
|
||||
// and embedded-<nav> enrichment (PR #4369 commit 4). Its output — the
|
||||
// grouped TOC and per-section fragment CFIs/sizes — must not depend on
|
||||
// whether the OPF/nav came from Rust or from zip.js.
|
||||
const navJs = await computeBookNav(jsBook);
|
||||
const navNative = await computeBookNav(nativeBook);
|
||||
expect(tocBrief(navNative.toc)).toEqual(tocBrief(navJs.toc));
|
||||
expect(Object.keys(navNative.sections).sort()).toEqual(Object.keys(navJs.sections).sort());
|
||||
expect(navFragmentMap(navNative)).toEqual(navFragmentMap(navJs));
|
||||
});
|
||||
}
|
||||
});
|
||||
@@ -4,8 +4,9 @@ import { useTranslation } from '@/hooks/useTranslation';
|
||||
import { useBookDataStore } from '@/store/bookDataStore';
|
||||
import { useSettingsStore } from '@/store/settingsStore';
|
||||
import { throttle } from '@/utils/throttle';
|
||||
import { getCoverFilename } from '@/utils/book';
|
||||
import { getCoverFilename, getLocalBookFilename } from '@/utils/book';
|
||||
import { eventDispatcher } from '@/utils/event';
|
||||
import { isTauriAppPlatform } from '@/services/environment';
|
||||
|
||||
export const useBookCoverAutoSave = (bookKey: string) => {
|
||||
const _ = useTranslation();
|
||||
@@ -22,20 +23,33 @@ export const useBookCoverAutoSave = (bookKey: string) => {
|
||||
const savedBookHash = settings.savedBookCoverForLockScreen;
|
||||
const savedCoverPath = settings.savedBookCoverForLockScreenPath;
|
||||
if (appService && book && savedBookHash && savedBookHash !== book?.hash) {
|
||||
const coverPath = await appService.resolveFilePath(getCoverFilename(book), 'Books');
|
||||
try {
|
||||
const lastCoverFilename = 'last-book-cover.png';
|
||||
const builtinImagesPath = await appService.resolveFilePath('', 'Images');
|
||||
if (!savedCoverPath || savedCoverPath === builtinImagesPath) {
|
||||
await appService.copyFile(coverPath, 'None', lastCoverFilename, 'Images');
|
||||
} else {
|
||||
await appService.copyFile(
|
||||
coverPath,
|
||||
'None',
|
||||
`${savedCoverPath}/${lastCoverFilename}`,
|
||||
'None',
|
||||
);
|
||||
const useBuiltinDest = !savedCoverPath || savedCoverPath === builtinImagesPath;
|
||||
|
||||
const wroteFullCover = await tryWriteFullCoverFromBook(
|
||||
appService,
|
||||
book,
|
||||
lastCoverFilename,
|
||||
useBuiltinDest ? null : savedCoverPath,
|
||||
);
|
||||
if (!wroteFullCover) {
|
||||
// Fallback: copy the on-disk thumbnail (still a valid PNG/JPEG
|
||||
// payload — webview / system image loaders sniff by header).
|
||||
const coverPath = await appService.resolveFilePath(getCoverFilename(book), 'Books');
|
||||
if (useBuiltinDest) {
|
||||
await appService.copyFile(coverPath, 'None', lastCoverFilename, 'Images');
|
||||
} else {
|
||||
await appService.copyFile(
|
||||
coverPath,
|
||||
'None',
|
||||
`${savedCoverPath}/${lastCoverFilename}`,
|
||||
'None',
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
settings.savedBookCoverForLockScreen = book.hash;
|
||||
useSettingsStore.getState().setSettings(settings);
|
||||
useSettingsStore.getState().saveSettings(envConfig, settings);
|
||||
@@ -61,3 +75,105 @@ export const useBookCoverAutoSave = (bookKey: string) => {
|
||||
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||
}, []);
|
||||
};
|
||||
|
||||
interface RustRawCoverImage {
|
||||
bytes: number[] | Uint8Array;
|
||||
mime: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Try to extract the full-resolution cover from the original book file via
|
||||
* the Rust native parsers and write it to the lock-screen target. Dispatches
|
||||
* by `book.format`:
|
||||
*
|
||||
* - EPUB → `extract_epub_cover_full` (zip-extract the manifest cover-image
|
||||
* entry, raw bytes, no resize),
|
||||
* - MOBI / AZW / AZW3 → `extract_mobi_cover_full` (re-run the EXTH 201/202
|
||||
* cover lookup, raw image-record bytes, no resize).
|
||||
*
|
||||
* Other formats (PDF, FB2, CBZ, …) don't have a Rust full-cover extractor
|
||||
* yet — we return `false` so the caller falls back to the on-disk thumbnail
|
||||
* (which, for those formats, is the only artwork we have).
|
||||
*
|
||||
* Returns true on success, false when the native path is unavailable, the
|
||||
* format isn't supported, or the command failed (caller falls back to the
|
||||
* on-disk thumbnail).
|
||||
*/
|
||||
async function tryWriteFullCoverFromBook(
|
||||
appService: ReturnType<typeof useEnv>['appService'],
|
||||
book: {
|
||||
format: string;
|
||||
hash: string;
|
||||
title: string;
|
||||
sourceTitle?: string;
|
||||
/**
|
||||
* For in-place imports the book bytes live outside `Books/<hash>/` —
|
||||
* we keep the user-supplied path on the Book record and must resolve
|
||||
* against it (with base `None`) rather than the synthetic
|
||||
* `Books/<hash>/<title>.<ext>` path `getLocalBookFilename` builds.
|
||||
* Mirrors the same in-place handling in `useWebDAVSync.pushBookFileNow`
|
||||
* and `cloudService.uploadBook`.
|
||||
*/
|
||||
filePath?: string;
|
||||
},
|
||||
destFilename: string,
|
||||
externalDestDir: string | null,
|
||||
): Promise<boolean> {
|
||||
if (!appService) return false;
|
||||
if (!isTauriAppPlatform()) return false;
|
||||
const command = pickFullCoverCommand(book.format);
|
||||
if (!command) return false;
|
||||
try {
|
||||
const { invoke } = await import('@tauri-apps/api/core');
|
||||
// In-place books point at a file the user owns elsewhere (Downloads,
|
||||
// an external SD card, …) — resolve against (filePath, 'None'). Hash-
|
||||
// copy books live under Books/<hash>/ with a name derived from their
|
||||
// metadata title; fall back to `getLocalBookFilename` for those. Same
|
||||
// base-dir dispatch as the sync push paths so behaviour stays uniform
|
||||
// across in-place / hash-copy imports.
|
||||
const localPath = book.filePath
|
||||
? await appService.resolveFilePath(book.filePath, 'None')
|
||||
: await appService.resolveFilePath(
|
||||
getLocalBookFilename(book as Parameters<typeof getLocalBookFilename>[0]),
|
||||
'Books',
|
||||
);
|
||||
const raw = await invoke<RustRawCoverImage>(command, {
|
||||
filePath: localPath,
|
||||
});
|
||||
const bytes = raw.bytes instanceof Uint8Array ? raw.bytes : new Uint8Array(raw.bytes);
|
||||
// BaseAppService.writeFile accepts ArrayBuffer; slice into a fresh
|
||||
// ArrayBuffer (not ArrayBufferLike) to satisfy the lib.dom typings.
|
||||
const ab = bytes.buffer.slice(
|
||||
bytes.byteOffset,
|
||||
bytes.byteOffset + bytes.byteLength,
|
||||
) as ArrayBuffer;
|
||||
if (externalDestDir) {
|
||||
await appService.writeFile(`${externalDestDir}/${destFilename}`, 'None', ab);
|
||||
} else {
|
||||
await appService.writeFile(destFilename, 'Images', ab);
|
||||
}
|
||||
return true;
|
||||
} catch (err) {
|
||||
console.warn('[useAutoSaveBookCover] full-cover extract failed, falling back:', err);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Map a `Book.format` to the matching Rust full-cover Tauri command, or
|
||||
* `null` if no native extractor exists for the format. Kept as a small
|
||||
* pure helper so the dispatch table stays in one place — adding a new
|
||||
* format (e.g. PDF) only needs a one-line addition here.
|
||||
*/
|
||||
function pickFullCoverCommand(format: string): string | null {
|
||||
switch (format) {
|
||||
case 'EPUB':
|
||||
return 'extract_epub_cover_full';
|
||||
case 'MOBI':
|
||||
case 'AZW3':
|
||||
case 'AZW':
|
||||
return 'extract_mobi_cover_full';
|
||||
default:
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -116,11 +116,25 @@ export const MIMETYPES: Record<BookFormat, string[]> = {
|
||||
MD: ['text/markdown', 'text/x-markdown'],
|
||||
};
|
||||
|
||||
export interface DocumentLoaderOptions {
|
||||
/**
|
||||
* Absolute filesystem path of `file`, used by Tauri builds to invoke the
|
||||
* Rust EPUB pre-parser (`parse_epub_full`). When omitted (web platform,
|
||||
* synthetic File, tests) the loader silently falls back to the
|
||||
* zip.js-only path. Callers SHOULD pass it whenever they have one --
|
||||
* the foliate-js init() drops from ~1.5s to ~0.3s on iOS for a typical
|
||||
* EPUB when the prefetch cache is hit.
|
||||
*/
|
||||
nativeFilePath?: string;
|
||||
}
|
||||
|
||||
export class DocumentLoader {
|
||||
private file: File;
|
||||
private nativeFilePath?: string;
|
||||
|
||||
constructor(file: File) {
|
||||
constructor(file: File, options: DocumentLoaderOptions = {}) {
|
||||
this.file = file;
|
||||
this.nativeFilePath = options.nativeFilePath;
|
||||
}
|
||||
|
||||
private async isZip(): Promise<boolean> {
|
||||
@@ -171,7 +185,10 @@ export class DocumentLoader {
|
||||
);
|
||||
}
|
||||
|
||||
private async makeZipLoader() {
|
||||
private async makeZipLoader(prefetch?: {
|
||||
textCache?: Map<string, string>;
|
||||
sizes?: Map<string, number>;
|
||||
}) {
|
||||
const getComment = async (): Promise<string | null> => {
|
||||
const EOCD_SIGNATURE = [0x50, 0x4b, 0x05, 0x06];
|
||||
const maxEOCDSearch = 1024 * 64;
|
||||
@@ -221,13 +238,69 @@ export class DocumentLoader {
|
||||
return entry ? f(entry, ...args) : null;
|
||||
};
|
||||
|
||||
const loadText = load((entry: Entry) =>
|
||||
const zipLoadText = load((entry: Entry) =>
|
||||
!entry.directory ? entry.getData(new TextWriter()) : null,
|
||||
);
|
||||
const loadBlob = load((entry: Entry, type?: string) =>
|
||||
!entry.directory ? entry.getData(new BlobWriter(type!)) : null,
|
||||
);
|
||||
const getSize = (name: string) => getEntry(name)?.uncompressedSize ?? 0;
|
||||
|
||||
// Prefetch fast-path: foliate-js's EPUB.init() reads container.xml,
|
||||
// the OPF, the EPUB3 nav and (if present) the NCX via this very
|
||||
// `loadText`. On Tauri we already have those bytes in memory from
|
||||
// the Rust `parse_epub_full` command, so we hand them back without
|
||||
// touching zip.js. Anything not in the cache falls through to the
|
||||
// original zip.js path (CSS/HTML/font assets the reader pulls
|
||||
// lazily as the user actually reads stay on the slow path, which
|
||||
// is fine -- they're also tiny per-call and async).
|
||||
const textCache = prefetch?.textCache;
|
||||
const sizesOverride = prefetch?.sizes;
|
||||
|
||||
// In-flight dedupe for spine-text loads.
|
||||
//
|
||||
// foliate-js's `Section` exposes both `loadText()` and `createDocument()`
|
||||
// (which internally re-runs `loadText` + parseFromString). Our nav
|
||||
// pipeline (`computeBookNav` and `enrichTocFromNavElements`) needs both
|
||||
// the raw HTML (for byte-size math + regex-based fragment locator) and
|
||||
// the parsed Document (for CFI computation), so it ends up calling them
|
||||
// back-to-back on the same href — without dedupe, every chapter pays for
|
||||
// two zip.js inflate calls per `computeBookNav`. On iOS WebView a 100KB
|
||||
// chapter inflate is ~3-5ms, so for a 100-section book this costs
|
||||
// ~300-500ms per first open. The dedupe is a single Map lookup on the
|
||||
// hot path, so the overhead when nothing is in flight is negligible.
|
||||
//
|
||||
// We intentionally only dedupe *concurrent* requests: as soon as the
|
||||
// promise settles, we drop it from the map so we don't retain inflated
|
||||
// chapter strings in memory (a long book is megabytes of text). This is
|
||||
// safe because the only consumer that cares about reuse — nav
|
||||
// computation — issues both calls in the same microtask span.
|
||||
const inflight = new Map<string, Promise<string | null>>();
|
||||
const dedupedZipLoadText = (name: string, ...args: [string?]): Promise<string | null> => {
|
||||
const existing = inflight.get(name);
|
||||
if (existing) return existing;
|
||||
const p =
|
||||
(zipLoadText(name, ...args) as Promise<string | null> | null) ?? Promise.resolve(null);
|
||||
const wrapped = Promise.resolve(p).finally(() => {
|
||||
// Release as soon as the promise settles; subsequent independent
|
||||
// reads will re-inflate (intentional — we don't want a nav-time
|
||||
// cache to hold the whole book in RAM).
|
||||
if (inflight.get(name) === wrapped) inflight.delete(name);
|
||||
});
|
||||
inflight.set(name, wrapped);
|
||||
return wrapped;
|
||||
};
|
||||
|
||||
const loadText = textCache
|
||||
? (name: string, ...args: [string?]) => {
|
||||
const cached = textCache.get(name);
|
||||
if (cached !== undefined) return Promise.resolve(cached);
|
||||
return dedupedZipLoadText(name, ...args);
|
||||
}
|
||||
: dedupedZipLoadText;
|
||||
|
||||
const getSize = sizesOverride
|
||||
? (name: string) => sizesOverride.get(name) ?? getEntry(name)?.uncompressedSize ?? 0
|
||||
: (name: string) => getEntry(name)?.uncompressedSize ?? 0;
|
||||
|
||||
return { entries, loadText, loadBlob, getSize, getComment, sha1: undefined };
|
||||
}
|
||||
@@ -261,7 +334,20 @@ export class DocumentLoader {
|
||||
}
|
||||
try {
|
||||
if (await this.isZip()) {
|
||||
const loader = await this.makeZipLoader();
|
||||
// EPUB-only fast path: ask Rust to pre-read OPF/nav/ncx + sizes.
|
||||
// CBZ/FBZ skip this -- they have no OPF and Rust has no parser
|
||||
// for them. We probe `isEPUBLike()` (= isZip but not CBZ/FBZ)
|
||||
// so the prefetch RPC only fires when it can actually be used.
|
||||
const isEPUBLike = !this.isCBZ() && !this.isFBZ();
|
||||
let prefetch: { textCache: Map<string, string>; sizes: Map<string, number> } | undefined;
|
||||
if (isEPUBLike && this.nativeFilePath) {
|
||||
const { tryNativePrefetchEpub } = await import('@/utils/tauriEpubBridge');
|
||||
const native = await tryNativePrefetchEpub(this.nativeFilePath);
|
||||
if (native) {
|
||||
prefetch = { textCache: native.textCache, sizes: native.sizes };
|
||||
}
|
||||
}
|
||||
const loader = await this.makeZipLoader(prefetch);
|
||||
const { entries } = loader;
|
||||
|
||||
if (this.isCBZ()) {
|
||||
|
||||
@@ -389,6 +389,10 @@ export abstract class BaseAppService implements AppService {
|
||||
return BookSvc.loadBookContent(this.fs, book);
|
||||
}
|
||||
|
||||
async resolveNativeBookFilePath(book: Book): Promise<string | null> {
|
||||
return BookSvc.resolveNativeBookFilePath(this.fs, this.resolveFilePath.bind(this), book);
|
||||
}
|
||||
|
||||
async loadBookConfig(book: Book, settings: SystemSettings): Promise<BookConfig> {
|
||||
return BookSvc.loadBookConfig(this.fs, book, settings);
|
||||
}
|
||||
|
||||
@@ -26,6 +26,8 @@ import type { BookNav } from '@/services/nav';
|
||||
import { partialMD5, md5 } from '@/utils/md5';
|
||||
import { getBaseFilename, getFilename } from '@/utils/path';
|
||||
import { BookDoc, DocumentLoader } from '@/libs/document';
|
||||
import { tryNativeParseEpub } from '@/utils/tauriEpubBridge';
|
||||
import { tryNativeParseMobi } from '@/utils/tauriMobiBridge';
|
||||
import { isPseStreamFileName, openPseStreamBook, parsePseStreamFileName } from './opds/pseStream';
|
||||
import { DEFAULT_BOOK_SEARCH_CONFIG, DEFAULT_FIXED_LAYOUT_VIEW_SETTINGS } from './constants';
|
||||
import { isContentURI, isValidURL, makeSafeFilename } from '@/utils/misc';
|
||||
@@ -251,6 +253,10 @@ export async function importBook(
|
||||
let format: BookFormat;
|
||||
let filename: string;
|
||||
let fileobj: File | undefined;
|
||||
// When the Rust EPUB parser succeeds it gives us the partialMD5 for free,
|
||||
// so we can short-circuit the JS hashing pass below.
|
||||
let nativeHash: string | undefined;
|
||||
let usedNativeParser = false;
|
||||
|
||||
if (transient && typeof file !== 'string') {
|
||||
throw new Error('Transient import is only supported for file paths');
|
||||
@@ -276,7 +282,47 @@ export async function importBook(
|
||||
if (!fileobj || fileobj.size === 0) {
|
||||
throw new Error('Invalid or empty book file');
|
||||
}
|
||||
({ book: loadedBook, format } = await new DocumentLoader(fileobj).open());
|
||||
// Q1 fast path: when running under Tauri with a real file
|
||||
// path, let Rust contribute the mechanical parts of the
|
||||
// import work — partialMD5 over the file, the downscaled
|
||||
// cover, and (for EPUB) the raw OPF bytes. Metadata
|
||||
// extraction itself runs through foliate-js so the import
|
||||
// path produces the same `Book.metadata` shape the reader
|
||||
// path does (`refines` chains / ONIX5 / language maps / EPUB
|
||||
// `belongs-to-collection` for EPUB; PalmDB UID identifier
|
||||
// for MOBI), without any `DocumentLoader.open()` overhead —
|
||||
// the importer never reads sections / toc / fixed-layout
|
||||
// detection, so spending CPU on a zip central-directory
|
||||
// scan, nav/ncx inflate, or PDB record-table walk would be
|
||||
// pure waste here.
|
||||
//
|
||||
// Both bridges are no-ops on web / non-eligible paths, so
|
||||
// the cost when neither matches is just two cheap regex
|
||||
// tests.
|
||||
let nativeBookDoc: BookDoc | undefined;
|
||||
let nativeFormat: BookFormat | undefined;
|
||||
if (typeof file === 'string' && !/\.txt$/i.test(filename)) {
|
||||
const nativeEpub = await tryNativeParseEpub(file);
|
||||
if (nativeEpub) {
|
||||
nativeBookDoc = nativeEpub.bookDoc;
|
||||
nativeFormat = 'EPUB' as BookFormat;
|
||||
nativeHash = nativeEpub.partialMd5;
|
||||
} else {
|
||||
const nativeMobi = await tryNativeParseMobi(file, fileobj);
|
||||
if (nativeMobi) {
|
||||
nativeBookDoc = nativeMobi.bookDoc;
|
||||
nativeFormat = nativeMobi.format;
|
||||
nativeHash = nativeMobi.partialMd5;
|
||||
}
|
||||
}
|
||||
}
|
||||
if (nativeBookDoc && nativeFormat) {
|
||||
loadedBook = nativeBookDoc;
|
||||
format = nativeFormat;
|
||||
usedNativeParser = true;
|
||||
} else {
|
||||
({ book: loadedBook, format } = await new DocumentLoader(fileobj).open());
|
||||
}
|
||||
}
|
||||
if (!loadedBook) {
|
||||
throw new Error('Unsupported or corrupted book file');
|
||||
@@ -290,7 +336,11 @@ export async function importBook(
|
||||
throw new Error(`Failed to open the book file: ${(error as Error).message || error}`);
|
||||
}
|
||||
|
||||
const hash = isPseStream ? md5(file as string) : await partialMD5(fileobj!);
|
||||
const hash = isPseStream
|
||||
? md5(file as string)
|
||||
: usedNativeParser
|
||||
? nativeHash!
|
||||
: await partialMD5(fileobj!);
|
||||
|
||||
const metaHash = getMetadataHash(loadedBook.metadata);
|
||||
let existingBook = lookupIndex
|
||||
@@ -415,7 +465,8 @@ export async function importBook(
|
||||
} catch {}
|
||||
}
|
||||
if (cover) {
|
||||
await fs.writeFile(getCoverFilename(book), 'Books', await cover.arrayBuffer());
|
||||
const coverBytes = await cover.arrayBuffer();
|
||||
await fs.writeFile(getCoverFilename(book), 'Books', coverBytes);
|
||||
}
|
||||
}
|
||||
// Never overwrite the config file only when it's not existed
|
||||
@@ -531,6 +582,30 @@ export async function loadBookContent(fs: FileSystem, book: Book): Promise<BookC
|
||||
return { book, file };
|
||||
}
|
||||
|
||||
/**
|
||||
* Best-effort resolution of an absolute, on-disk filesystem path for a book.
|
||||
*
|
||||
* Returns null when the book is not stored on disk (e.g. in-memory blob,
|
||||
* remote URL) or the path cannot be resolved. The returned path is
|
||||
* suitable for handing to native (Rust) commands that read the file
|
||||
* directly via std::fs.
|
||||
*/
|
||||
export async function resolveNativeBookFilePath(
|
||||
fs: FileSystem,
|
||||
resolveFilePath: (path: string, base: BaseDir) => Promise<string>,
|
||||
book: Book,
|
||||
): Promise<string | null> {
|
||||
try {
|
||||
const source = await resolveBookContentSource(fs, book);
|
||||
if (source.kind !== 'managed' && source.kind !== 'external') return null;
|
||||
const fp = await resolveFilePath(source.path, source.base);
|
||||
if (!fp) return null;
|
||||
return fp.startsWith('file://') ? decodeURI(fp.slice('file://'.length)) : fp;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
export async function loadBookConfig(
|
||||
fs: FileSystem,
|
||||
book: Book,
|
||||
|
||||
@@ -104,31 +104,52 @@ export const enrichTocFromNavElements = async (
|
||||
|
||||
const sectionIdSet = new Set(sections.map((s) => s.id));
|
||||
|
||||
// Two-phase scan:
|
||||
// 1) Concurrently `loadText` every section and quick-filter on
|
||||
// `content.includes('<nav')` — cheap (ASCII substring on the raw
|
||||
// HTML) and immediately discards 90%+ of chapters that have no
|
||||
// embedded nav. Concurrency lets the zip inflates overlap.
|
||||
// 2) For survivors, concurrently `createDocument` + walk `<nav>`
|
||||
// elements. The makeZipLoader dedupes in-flight loadText calls by
|
||||
// name, so phase 2's `createDocument()` reuses phase 1's inflated
|
||||
// bytes when the two awaits overlap.
|
||||
type Survivor = { section: (typeof sections)[number] };
|
||||
const survivors: Survivor[] = [];
|
||||
const phase1 = await Promise.all(
|
||||
sections.map(async (section) => {
|
||||
if (!section.loadText) return null;
|
||||
try {
|
||||
const content = await section.loadText();
|
||||
if (!content || !content.includes('<nav')) return null;
|
||||
return { section };
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}),
|
||||
);
|
||||
for (const s of phase1) if (s) survivors.push(s);
|
||||
|
||||
const phase2 = await Promise.all(
|
||||
survivors.map(async ({ section }) => {
|
||||
let doc: Document;
|
||||
try {
|
||||
doc = await section.createDocument();
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
const navs = Array.from(doc.getElementsByTagName('nav'));
|
||||
if (navs.length === 0) return null;
|
||||
return { sectionId: section.id, navs };
|
||||
}),
|
||||
);
|
||||
|
||||
const collected: ExtractedNavItem[] = [];
|
||||
const seenHref = new Set<string>();
|
||||
|
||||
for (const section of sections) {
|
||||
if (!section.loadText) continue;
|
||||
let content: string | null = null;
|
||||
try {
|
||||
content = await section.loadText();
|
||||
} catch {
|
||||
content = null;
|
||||
}
|
||||
if (!content || !content.includes('<nav')) continue;
|
||||
|
||||
let doc: Document;
|
||||
try {
|
||||
doc = await section.createDocument();
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
|
||||
const navs = Array.from(doc.getElementsByTagName('nav'));
|
||||
if (navs.length === 0) continue;
|
||||
|
||||
for (const result of phase2) {
|
||||
if (!result) continue;
|
||||
const { sectionId, navs } = result;
|
||||
for (const navEl of navs) {
|
||||
const extracted = extractTocFromNav(navEl, section.id);
|
||||
const extracted = extractTocFromNav(navEl, sectionId);
|
||||
for (const item of extracted) {
|
||||
const sectionPart = hrefSection(item.href);
|
||||
if (!sectionPart || !sectionIdSet.has(sectionPart)) continue;
|
||||
|
||||
@@ -13,14 +13,32 @@ const findFragmentPosition = (html: string, fragmentId: string | undefined): num
|
||||
return -1;
|
||||
};
|
||||
|
||||
// Memoized variant for use inside buildSectionFragments — within a single
|
||||
// section, each TOC fragment id is consulted twice (once as the current
|
||||
// boundary, once as the next item's `prev`), so without caching we run 2N
|
||||
// regex scans over the whole HTML for N fragments. The cache collapses that
|
||||
// to N. Cache lifetime is one section pass; not exposed.
|
||||
const makeFragmentPositionCache = (html: string) => {
|
||||
const memo = new Map<string, number>();
|
||||
return (fragmentId: string | undefined): number => {
|
||||
if (!fragmentId) return html.length;
|
||||
const cached = memo.get(fragmentId);
|
||||
if (cached !== undefined) return cached;
|
||||
const pos = findFragmentPosition(html, fragmentId);
|
||||
memo.set(fragmentId, pos);
|
||||
return pos;
|
||||
};
|
||||
};
|
||||
|
||||
const calculateFragmentSize = (
|
||||
content: string,
|
||||
fragmentId: string | undefined,
|
||||
prevFragmentId: string | undefined,
|
||||
positionOf: (id: string | undefined) => number,
|
||||
): number => {
|
||||
const endPos = findFragmentPosition(content, fragmentId);
|
||||
const endPos = positionOf(fragmentId);
|
||||
if (endPos < 0) return 0;
|
||||
const startPos = prevFragmentId ? findFragmentPosition(content, prevFragmentId) : 0;
|
||||
const startPos = prevFragmentId ? positionOf(prevFragmentId) : 0;
|
||||
const validStartPos = Math.max(0, startPos);
|
||||
if (endPos < validStartPos) return 0;
|
||||
return new Blob([content.substring(validStartPos, endPos)]).size;
|
||||
@@ -36,10 +54,50 @@ type CFIModule = {
|
||||
fromElements: (elements: Element[]) => string[];
|
||||
};
|
||||
|
||||
// foliate-js's `fromElements` walks `parentNode` chain via repeated calls to
|
||||
// `nodeToParts(parentNode)`. The termination check (epubcfi.js line 281) is
|
||||
// "stop when current node's parentNode === documentElement", which means
|
||||
// recursion stops AT `<body>` (parent is `<html>`/documentElement) but NOT at
|
||||
// `<html>` itself (parent is the Document, which !== documentElement, so it
|
||||
// recurses one more level into the Document, then tries indexChildNodes on
|
||||
// Document.parentNode === null → "null is not an object (evaluating
|
||||
// 'node.childNodes')").
|
||||
//
|
||||
// Concretely it blows up when:
|
||||
// - element is `documentElement` itself,
|
||||
// - element is `<body>` itself (recursion starts at `<html>` and overshoots),
|
||||
// - element is detached / parentNode === null,
|
||||
// - element lives outside <body> (e.g. ids on <head>) — even when it
|
||||
// wouldn't crash, the CFI it would produce is meaningless for our use.
|
||||
//
|
||||
// Reject these up-front so we silently fall back to the section CFI instead
|
||||
// of throwing + spamming console.warn for every fragment.
|
||||
const isCfiAddressable = (element: Element): boolean => {
|
||||
const doc = element.ownerDocument;
|
||||
if (!doc) return false;
|
||||
if (element === doc.documentElement) return false;
|
||||
const body = doc.body;
|
||||
if (!body) return false;
|
||||
// Must be a STRICT descendant of <body>. Body itself overshoots the
|
||||
// foliate-js termination check (see comment above).
|
||||
if (element === body) return false;
|
||||
if (!body.contains(element)) return false;
|
||||
// Defensive: parentNode chain must reach <body> without hitting null first.
|
||||
// Covers detached subtrees and weird DOMs where contains() lies.
|
||||
let cursor: Node | null = element.parentNode;
|
||||
while (cursor && cursor !== body) {
|
||||
cursor = cursor.parentNode;
|
||||
}
|
||||
return cursor === body;
|
||||
};
|
||||
|
||||
const buildFragmentCfi = (section: SectionItem, element: Element | null): string => {
|
||||
const cfiLib = CFI as unknown as CFIModule;
|
||||
if (!element || !isCfiAddressable(element)) {
|
||||
return section.cfi;
|
||||
}
|
||||
try {
|
||||
const rel = element ? (cfiLib.fromElements([element])[0] ?? '') : '';
|
||||
const rel = cfiLib.fromElements([element])[0] ?? '';
|
||||
return cfiLib.joinIndir(section.cfi, rel);
|
||||
} catch (e) {
|
||||
console.warn('Failed to build CFI for fragment, falling back to section CFI:', e);
|
||||
@@ -56,6 +114,7 @@ export const buildSectionFragments = (
|
||||
splitHref: (href: string) => Array<string | number>,
|
||||
): SectionFragment[] => {
|
||||
const out: SectionFragment[] = [];
|
||||
const positionOf = makeFragmentPositionCache(content);
|
||||
for (let i = 0; i < fragments.length; i++) {
|
||||
const fragment = fragments[i]!;
|
||||
const [, rawFragmentId] = splitHref(fragment.href) as [string | undefined, string | undefined];
|
||||
@@ -69,7 +128,7 @@ export const buildSectionFragments = (
|
||||
|
||||
const element = getHTMLFragmentElement(doc, fragmentId);
|
||||
const cfi = buildFragmentCfi(section, element);
|
||||
const size = calculateFragmentSize(content, fragmentId, prevFragmentId);
|
||||
const size = calculateFragmentSize(content, fragmentId, prevFragmentId, positionOf);
|
||||
|
||||
out.push({
|
||||
id: fragment.href,
|
||||
|
||||
@@ -97,33 +97,51 @@ export const computeBookNav = async (bookDoc: BookDoc): Promise<BookNav> => {
|
||||
const groups = groupItemsBySection(bookDoc, allItems);
|
||||
const splitHref = (href: string) => bookDoc.splitTOCHref(href);
|
||||
|
||||
for (const [sectionId, { base, fragments }] of groups.entries()) {
|
||||
const section = sectionMap.get(sectionId);
|
||||
if (!section || fragments.length === 0) continue;
|
||||
if (!section.loadText) continue;
|
||||
// Process sections concurrently. Each section's work is independent: the
|
||||
// only shared writes are into `sections` (keyed by sectionId, no
|
||||
// collisions) and into the local `bookSections` later (after all promises
|
||||
// resolve). Concurrency here lets the zip-text inflate calls overlap with
|
||||
// each other and with the `parseFromString` of earlier sections — on iOS
|
||||
// WebView this turns the previously-serial chapter loop into something
|
||||
// bounded by the slowest single inflate + parse rather than their sum.
|
||||
//
|
||||
// We don't bound concurrency: a typical EPUB has ≲ 200 sections and the
|
||||
// zip.js loader is happy to interleave them; if memory pressure ever
|
||||
// matters we can drop in a small p-limit-style throttle.
|
||||
const sectionResults = await Promise.all(
|
||||
Array.from(groups.entries()).map(async ([sectionId, { base, fragments }]) => {
|
||||
const section = sectionMap.get(sectionId);
|
||||
if (!section || fragments.length === 0) return null;
|
||||
if (!section.loadText) return null;
|
||||
|
||||
const content = await section.loadText();
|
||||
if (!content) continue;
|
||||
// Issue both the raw-text and DOM-parse loads concurrently. The
|
||||
// makeZipLoader now dedupes in-flight loadText calls by name, so the
|
||||
// two awaits below resolve from a single zip inflate per section
|
||||
// instead of two.
|
||||
const contentP = section.loadText();
|
||||
const docP = section.createDocument().catch((e: unknown) => {
|
||||
console.warn(`Failed to parse section ${sectionId} for fragment CFIs:`, e);
|
||||
return null;
|
||||
});
|
||||
const [content, doc] = await Promise.all([contentP, docP]);
|
||||
if (!content) return null;
|
||||
if (!doc) return null;
|
||||
|
||||
let doc: Document | null = null;
|
||||
try {
|
||||
doc = await section.createDocument();
|
||||
} catch (e) {
|
||||
console.warn(`Failed to parse section ${sectionId} for fragment CFIs:`, e);
|
||||
}
|
||||
if (!doc) continue;
|
||||
const sectionFragments = buildSectionFragments(
|
||||
section,
|
||||
fragments,
|
||||
base,
|
||||
content,
|
||||
doc,
|
||||
splitHref,
|
||||
);
|
||||
if (sectionFragments.length === 0) return null;
|
||||
return [sectionId, { id: sectionId, fragments: sectionFragments }] as const;
|
||||
}),
|
||||
);
|
||||
|
||||
const sectionFragments = buildSectionFragments(
|
||||
section,
|
||||
fragments,
|
||||
base,
|
||||
content,
|
||||
doc,
|
||||
splitHref,
|
||||
);
|
||||
if (sectionFragments.length > 0) {
|
||||
sections[sectionId] = { id: sectionId, fragments: sectionFragments };
|
||||
}
|
||||
for (const r of sectionResults) {
|
||||
if (r) sections[r[0]] = r[1];
|
||||
}
|
||||
|
||||
// Attach the freshly computed fragments to live sections so the bake can see
|
||||
|
||||
@@ -181,7 +181,15 @@ export const useReaderStore = create<ReaderStore>((set, get) => ({
|
||||
} else {
|
||||
const content = (await appService.loadBookContent(book)) as BookContent;
|
||||
file = content.file;
|
||||
const doc = await new DocumentLoader(file).open();
|
||||
let nativeFilePath: string | null = null;
|
||||
try {
|
||||
nativeFilePath = await appService.resolveNativeBookFilePath(book);
|
||||
} catch (err) {
|
||||
console.warn('resolveNativeBookFilePath failed', err);
|
||||
}
|
||||
const doc = await new DocumentLoader(file, {
|
||||
nativeFilePath: nativeFilePath ?? undefined,
|
||||
}).open();
|
||||
bookDoc = doc.book;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -203,6 +203,7 @@ export interface AppService {
|
||||
loadBookNav(book: Book): Promise<BookNav | null>;
|
||||
saveBookNav(book: Book, nav: BookNav): Promise<void>;
|
||||
loadBookContent(book: Book): Promise<BookContent>;
|
||||
resolveNativeBookFilePath(book: Book): Promise<string | null>;
|
||||
loadLibraryBooks(): Promise<Book[]>;
|
||||
saveLibraryBooks(books: Book[]): Promise<void>;
|
||||
getCoverImageUrl(book: Book): string;
|
||||
|
||||
@@ -0,0 +1,255 @@
|
||||
// JS<->Rust EPUB bridge for Tauri targets.
|
||||
//
|
||||
// Architectural split:
|
||||
//
|
||||
// * Rust handles the *mechanical* zip work that's expensive on a
|
||||
// WebView: opening the zip + central-directory parse, partialMD5
|
||||
// over the file, locating the cover entry, decoding/resizing the
|
||||
// cover image, and (on the open hot path) prefetching nav/ncx
|
||||
// bytes + the entry-size map. It also forwards the OPF bytes it
|
||||
// already had to read for cover resolution.
|
||||
// * foliate-js stays the single source of truth for OPF metadata
|
||||
// extraction (title, author, identifier, language map, refines /
|
||||
// `belongs-to-collection` graph, ONIX5 codelists, …). The import
|
||||
// path runs foliate's `parseEpubMetadataFromXML` directly on the
|
||||
// OPF bytes Rust hands over — no zip.js central-directory scan,
|
||||
// no nav/ncx inflate, no spine traversal — keeping the import
|
||||
// hot path fast while ensuring `Book.metadata` stays byte-stable
|
||||
// against what the reader path produces.
|
||||
//
|
||||
// Two Tauri commands back this:
|
||||
//
|
||||
// * parse_epub_metadata — import path. Returns
|
||||
// `{ partialMd5, cover, coverMime, opfPath,
|
||||
// opfBytes }`. The bridge runs foliate's
|
||||
// OPF-only metadata extractor on
|
||||
// `opfBytes` and assembles a lightweight
|
||||
// BookDoc stub the importer consumes.
|
||||
// * parse_epub_full — open path. Returns OPF prefetch + nav/
|
||||
// ncx bytes + entry-size map for the
|
||||
// DocumentLoader on the reader hot path.
|
||||
//
|
||||
// Avoids ferrying multi-MB blobs across the JS<->Rust IPC boundary
|
||||
// and is a no-op on the web platform.
|
||||
import { invoke } from '@tauri-apps/api/core';
|
||||
import { isTauriAppPlatform } from '@/services/environment';
|
||||
import type { BookDoc, BookMetadata } from '@/libs/document';
|
||||
|
||||
// ─── shared helpers ──────────────────────────────────────────────────
|
||||
|
||||
const isEligibleEpubPath = (filePath: string | undefined): filePath is string =>
|
||||
!!filePath && isTauriAppPlatform() && /\.epub$/i.test(filePath);
|
||||
|
||||
/**
|
||||
* Convert a `Vec<u8>` returned by Rust over Tauri's IPC into a plain
|
||||
* `Uint8Array`. Depending on Tauri's serializer / WebView the wire form
|
||||
* is either a `number[]` or already a typed array — normalize both.
|
||||
*/
|
||||
const toUint8Array = (bytes: number[] | Uint8Array): Uint8Array =>
|
||||
bytes instanceof Uint8Array ? bytes : new Uint8Array(bytes);
|
||||
|
||||
/** Decode a UTF-8 byte buffer (Rust `Vec<u8>` over IPC) into a string. */
|
||||
const bytesArrayToString = (bytes: number[] | Uint8Array): string =>
|
||||
new TextDecoder('utf-8').decode(toUint8Array(bytes));
|
||||
|
||||
// ─── parse_epub_metadata (import path) ───────────────────────────────
|
||||
|
||||
interface RustParsedEpubMetadata {
|
||||
/** partialMD5 of the EPUB file. Same algorithm as utils/md5.ts::partialMD5. */
|
||||
partialMd5: string;
|
||||
/** Pre-resized cover bytes (Vec<u8> over IPC), or null/absent when the
|
||||
* EPUB has no cover. The Rust side runs the resize + JPEG re-encode
|
||||
* through the `image` crate, which is materially faster than a
|
||||
* `createImageBitmap` + canvas round-trip on Android mid-tier devices
|
||||
* during bulk imports. */
|
||||
cover?: number[] | Uint8Array | null;
|
||||
/** MIME of `cover` after the (optional) re-encode. Always paired with
|
||||
* `cover`. We propagate it so the JS-side SVG-cover branch in
|
||||
* `bookService.importBook` (which checks `cover.type === "image/svg+xml"`
|
||||
* to route through svg2png) keeps working even on the native fast path. */
|
||||
coverMime?: string | null;
|
||||
/** OPF zip path. Always present when `partialMd5` is. */
|
||||
opfPath: string;
|
||||
/** OPF bytes — Rust read these for cover resolution; we forward them
|
||||
* so the importer can run foliate's OPF metadata extractor without a
|
||||
* second zip access. */
|
||||
opfBytes: number[] | Uint8Array;
|
||||
}
|
||||
|
||||
export interface NativeParsedEpub {
|
||||
/** partialMD5 of the file, ready to use as the `Book.hash`. */
|
||||
partialMd5: string;
|
||||
/** Lightweight BookDoc stub: only `metadata` and `getCover()` are
|
||||
* populated, which is all `bookService.importBook` consults on the
|
||||
* import hot path. Sections / TOC / fixed-layout detection are
|
||||
* populated lazily by the reader when the user actually opens the
|
||||
* book (which goes through the regular `DocumentLoader` path). */
|
||||
bookDoc: BookDoc;
|
||||
}
|
||||
|
||||
/**
|
||||
* Build a BookDoc stub for the importer.
|
||||
*
|
||||
* `metadata` is whatever foliate-js's `parseEpubMetadataFromXML` returns
|
||||
* for the OPF Rust handed us — byte-stable against the reader path.
|
||||
* `getCover()` returns the Rust-downscaled blob; everything else is the
|
||||
* minimum the importer reads (it never touches `sections` / `toc` /
|
||||
* `splitTOCHref`).
|
||||
*/
|
||||
const buildBookDocStub = (metadata: BookMetadata, coverBlob: Blob | null): BookDoc => {
|
||||
const stub = {
|
||||
metadata,
|
||||
rendition: {},
|
||||
dir: 'ltr',
|
||||
toc: [],
|
||||
sections: [],
|
||||
splitTOCHref: () => [null, null],
|
||||
getCover: async () => coverBlob,
|
||||
} as unknown as BookDoc;
|
||||
return stub;
|
||||
};
|
||||
|
||||
/**
|
||||
* Try the import-time native fast-path. Returns `null` on web platform,
|
||||
* non-EPUB path, or any IPC / parse error so callers can fall back to
|
||||
* the regular foliate-js `DocumentLoader.open()` pipeline with no
|
||||
* behavioural change.
|
||||
*
|
||||
* On success the bridge:
|
||||
* 1. invokes the Rust `parse_epub_metadata` command (one IPC, one
|
||||
* zip open, one partialMD5 pass);
|
||||
* 2. parses the returned OPF bytes through foliate-js's exported
|
||||
* `parseEpubMetadataFromXML`, so the resulting `metadata` shape
|
||||
* (refines chains, ONIX5 codelists, language maps,
|
||||
* `belongs-to-collection`, …) matches the reader path exactly;
|
||||
* 3. wraps the cover bytes into a Blob carrying the Rust-supplied
|
||||
* MIME (so `bookService.importBook`'s `cover.type === 'image/svg+xml'`
|
||||
* branch can still route through svg2png).
|
||||
*/
|
||||
export const tryNativeParseEpub = async (
|
||||
filePath: string | undefined,
|
||||
): Promise<NativeParsedEpub | null> => {
|
||||
if (!isEligibleEpubPath(filePath)) return null;
|
||||
try {
|
||||
const rust = await invoke<RustParsedEpubMetadata>('parse_epub_metadata', {
|
||||
filePath,
|
||||
});
|
||||
if (!rust || !rust.partialMd5 || !rust.opfPath || !rust.opfBytes) return null;
|
||||
|
||||
// foliate-js exposes `parseEpubMetadataFromXML` so callers that
|
||||
// already have OPF bytes (us — Rust just read them out of the zip)
|
||||
// can derive `Book.metadata` without driving the full `EPUB.init()`
|
||||
// (which would force `@zip.js/zip.js` to scan the central directory
|
||||
// and inflate nav/ncx files the importer never reads). Dynamic
|
||||
// import keeps the bridge tree-shakable on web builds where this
|
||||
// path is never reached.
|
||||
const epubModule = (await import('foliate-js/epub.js')) as unknown as {
|
||||
parseEpubMetadataFromXML: (xml: string) => { metadata: BookMetadata };
|
||||
};
|
||||
const opfXml = bytesArrayToString(rust.opfBytes);
|
||||
const { metadata } = epubModule.parseEpubMetadataFromXML(opfXml);
|
||||
|
||||
let coverBlob: Blob | null = null;
|
||||
if (rust.cover && rust.coverMime) {
|
||||
const bytes = toUint8Array(rust.cover);
|
||||
if (bytes.byteLength > 0) {
|
||||
// Slice into a fresh ArrayBuffer to satisfy lib.dom Blob typings
|
||||
// (which require BlobPart = ArrayBuffer/ArrayBufferView<ArrayBuffer>,
|
||||
// not the ArrayBufferLike that the Uint8Array constructor exposes).
|
||||
const ab = bytes.buffer.slice(
|
||||
bytes.byteOffset,
|
||||
bytes.byteOffset + bytes.byteLength,
|
||||
) as ArrayBuffer;
|
||||
coverBlob = new Blob([ab], { type: rust.coverMime });
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
partialMd5: rust.partialMd5,
|
||||
bookDoc: buildBookDocStub(metadata, coverBlob),
|
||||
};
|
||||
} catch (err) {
|
||||
console.warn('[tauriEpubBridge] native parse failed, falling back to JS:', err);
|
||||
return null;
|
||||
}
|
||||
};
|
||||
|
||||
// ─── parse_epub_full (open hot-path prefetch) ────────────────────────
|
||||
|
||||
interface RustParsedEpubFull {
|
||||
partialMd5: string;
|
||||
opfPath: string;
|
||||
opfBytes: number[] | Uint8Array;
|
||||
navPath?: string | null;
|
||||
navBytes?: number[] | Uint8Array | null;
|
||||
ncxPath?: string | null;
|
||||
ncxBytes?: number[] | Uint8Array | null;
|
||||
/**
|
||||
* Map: zip entry name → uncompressed size in bytes. Sent over IPC as a
|
||||
* plain object (`{ "OEBPS/x.html": 12345, ... }`) and rehydrated into a
|
||||
* Map below for O(1) `getSize()` calls.
|
||||
*/
|
||||
sizes: Record<string, number>;
|
||||
}
|
||||
|
||||
export interface NativeEpubPrefetch {
|
||||
/**
|
||||
* Map of zip-path → text content. Populated for the OPF, EPUB3 nav doc,
|
||||
* NCX (if present), and a synthetic META-INF/container.xml that points
|
||||
* foliate-js at our OPF path. Anything not in the map falls through to
|
||||
* the regular zip.js loadText path.
|
||||
*/
|
||||
textCache: Map<string, string>;
|
||||
/** Map of zip-path → uncompressed byte size, for foliate-js getSize(). */
|
||||
sizes: Map<string, number>;
|
||||
/** partialMD5 of the file, returned alongside the prefetch in case the
|
||||
* caller wants to reuse it (e.g. to set Book.hash without rehashing). */
|
||||
partialMd5: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the minimal META-INF/container.xml that foliate-js's EPUB.init()
|
||||
* looks at to find the OPF. We synthesize this from `opfPath` so the JS
|
||||
* side never has to inflate the real container entry from the zip.
|
||||
*/
|
||||
const buildContainerXml = (opfPath: string): string =>
|
||||
`<?xml version="1.0" encoding="UTF-8"?>` +
|
||||
`<container version="1.0" xmlns="urn:oasis:names:tc:opendocument:xmlns:container">` +
|
||||
`<rootfiles>` +
|
||||
`<rootfile full-path="${opfPath}" media-type="application/oebps-package+xml"/>` +
|
||||
`</rootfiles>` +
|
||||
`</container>`;
|
||||
|
||||
/**
|
||||
* Try to prefetch OPF/nav/ncx + entry sizes for an EPUB via Rust on the
|
||||
* reader open hot path. Returns null when the native path is unavailable
|
||||
* (web platform, missing file path, IPC error) so the caller can fall
|
||||
* back to the regular zip.js-only DocumentLoader.
|
||||
*/
|
||||
export const tryNativePrefetchEpub = async (
|
||||
filePath: string | undefined,
|
||||
): Promise<NativeEpubPrefetch | null> => {
|
||||
if (!isEligibleEpubPath(filePath)) return null;
|
||||
try {
|
||||
const rust = await invoke<RustParsedEpubFull>('parse_epub_full', {
|
||||
filePath,
|
||||
});
|
||||
if (!rust || !rust.partialMd5 || !rust.opfPath || !rust.opfBytes) return null;
|
||||
|
||||
const textCache = new Map<string, string>();
|
||||
textCache.set('META-INF/container.xml', buildContainerXml(rust.opfPath));
|
||||
textCache.set(rust.opfPath, bytesArrayToString(rust.opfBytes));
|
||||
if (rust.navPath && rust.navBytes) {
|
||||
textCache.set(rust.navPath, bytesArrayToString(rust.navBytes));
|
||||
}
|
||||
if (rust.ncxPath && rust.ncxBytes) {
|
||||
textCache.set(rust.ncxPath, bytesArrayToString(rust.ncxBytes));
|
||||
}
|
||||
|
||||
const sizes = new Map<string, number>(Object.entries(rust.sizes ?? {}));
|
||||
return { textCache, sizes, partialMd5: rust.partialMd5 };
|
||||
} catch (err) {
|
||||
console.warn('[tauriEpubBridge] native prefetch failed, falling back to JS:', err);
|
||||
return null;
|
||||
}
|
||||
};
|
||||
@@ -0,0 +1,192 @@
|
||||
// JS<->Rust MOBI/AZW/AZW3 bridge for Tauri targets.
|
||||
//
|
||||
// Architectural split (mirrors `tauriEpubBridge`):
|
||||
//
|
||||
// * Rust handles the *mechanical* work that's expensive on a
|
||||
// WebView: `partialMD5` over the file, plus locating + decoding +
|
||||
// resizing the cover image. The cover bytes are downscaled by
|
||||
// the `image` crate, which is materially faster than a
|
||||
// `createImageBitmap` + canvas round-trip on Android mid-tier
|
||||
// devices during bulk imports.
|
||||
//
|
||||
// * foliate-js stays the single source of truth for MOBI metadata
|
||||
// extraction (title / author / identifier=PalmDB UID /
|
||||
// publisher / description / language / subjects / …). The
|
||||
// importer runs foliate-js's exported `readMobiMetadata` on the
|
||||
// same File so `Book.metadata.identifier` is byte-stable against
|
||||
// what the reader path produces (foliate `mobi.uid.toString()`).
|
||||
// Existing libraries don't get every MOBI re-imported as a
|
||||
// duplicate after the metaHash recomputes.
|
||||
//
|
||||
// `readMobiMetadata` deliberately stops at MOBI.open()'s
|
||||
// metadata-only short-circuit: PalmDB header + record offsets
|
||||
// table + record 0 (PalmDoc/MobiHeader/EXTH) + decoder setup,
|
||||
// and skips the MOBI6/KF8 init() that walks every text record.
|
||||
// Roughly the cost of a stub-style import path while preserving
|
||||
// foliate's metadata semantics.
|
||||
//
|
||||
// One Tauri command backs this:
|
||||
//
|
||||
// * parse_mobi_metadata — returns `{ partialMd5, cover? }`. The
|
||||
// bridge wraps `cover` into a Blob and
|
||||
// bolts it onto a BookDoc stub whose
|
||||
// metadata comes from foliate-js.
|
||||
//
|
||||
// Avoids ferrying multi-MB MOBI/AZW3 blobs across the JS<->Rust IPC
|
||||
// boundary and is a no-op on the web platform.
|
||||
import { invoke } from '@tauri-apps/api/core';
|
||||
import { isTauriAppPlatform } from '@/services/environment';
|
||||
import type { BookDoc, BookMetadata } from '@/libs/document';
|
||||
import type { BookFormat } from '@/types/book';
|
||||
|
||||
// ─── shared helpers ──────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Match every Kindle container we feed to the foliate-js MOBI loader on
|
||||
* the web fallback path: classic MOBI, Amazon's AZW (KF7), AZW3 (KF8),
|
||||
* and the legacy Mobipocket .prc wrapper.
|
||||
*/
|
||||
const MOBI_EXT_RE = /\.(mobi|azw|azw3|prc)$/i;
|
||||
|
||||
export const isEligibleMobiPath = (filePath: string | undefined): filePath is string =>
|
||||
!!filePath && isTauriAppPlatform() && MOBI_EXT_RE.test(filePath);
|
||||
|
||||
/**
|
||||
* Map the file's extension to the on-disk `Book.format`.
|
||||
*
|
||||
* `.azw3` is foliate's canonical "Kindle Format 8" container, `.azw`
|
||||
* is Amazon's wrapper around classic MOBI (KF7), `.mobi` and `.prc`
|
||||
* both mean classic Mobipocket. We honour the user-facing extension
|
||||
* here so the library list matches what the user dragged in, even
|
||||
* though foliate's MOBI loader doesn't differentiate at runtime.
|
||||
*/
|
||||
const inferMobiFormat = (filePath: string): BookFormat => {
|
||||
const ext = filePath.toLowerCase().split('.').pop();
|
||||
if (ext === 'azw3') return 'AZW3' as BookFormat;
|
||||
if (ext === 'azw') return 'AZW' as BookFormat;
|
||||
return 'MOBI' as BookFormat;
|
||||
};
|
||||
|
||||
// ─── parse_mobi_metadata (import path) ───────────────────────────────
|
||||
|
||||
interface RustRawCoverImage {
|
||||
/** Tauri's IPC serializer ships Vec<u8> as either a number[] or a typed
|
||||
* array; we accept either and normalize via `Uint8Array.from(...)`. */
|
||||
bytes: number[] | Uint8Array;
|
||||
mime: string;
|
||||
}
|
||||
|
||||
interface RustParsedMobi {
|
||||
partialMd5: string;
|
||||
cover?: RustRawCoverImage | null;
|
||||
}
|
||||
|
||||
export interface NativeParsedMobi {
|
||||
/** partialMD5 of the file, ready to use as the `Book.hash`. */
|
||||
partialMd5: string;
|
||||
/** Resolved on-disk format from the file extension (MOBI / AZW / AZW3). */
|
||||
format: BookFormat;
|
||||
/** Lightweight BookDoc stub: only `metadata` and `getCover()` are
|
||||
* populated, which is all `bookService.importBook` consults on the
|
||||
* import hot path. */
|
||||
bookDoc: BookDoc;
|
||||
}
|
||||
|
||||
/**
|
||||
* Build a BookDoc stub for the importer. `metadata` comes from
|
||||
* foliate-js's `readMobiMetadata` so it matches the reader path
|
||||
* byte-for-byte; `getCover()` returns the Rust-downscaled blob (or
|
||||
* falls back to foliate's `getCover` thunk when Rust didn't extract
|
||||
* a cover, which keeps the pre-Rust behaviour for cover-less files).
|
||||
*/
|
||||
const buildBookDocStub = (
|
||||
metadata: BookMetadata,
|
||||
coverBlob: Blob | null,
|
||||
foliateGetCover: () => Promise<Blob | null | undefined>,
|
||||
): BookDoc => {
|
||||
const stub = {
|
||||
metadata,
|
||||
rendition: {},
|
||||
dir: 'ltr',
|
||||
toc: [],
|
||||
sections: [],
|
||||
splitTOCHref: () => [null, null],
|
||||
getCover: async () => {
|
||||
if (coverBlob) return coverBlob;
|
||||
const fallback = await foliateGetCover();
|
||||
return fallback ?? null;
|
||||
},
|
||||
} as unknown as BookDoc;
|
||||
return stub;
|
||||
};
|
||||
|
||||
/**
|
||||
* Try the import-time native fast-path: ask Rust for the file's
|
||||
* partialMD5 + downscaled cover, then run foliate-js's exported
|
||||
* `readMobiMetadata` on the supplied `File` to derive metadata.
|
||||
* Returns `null` on web platform / non-MOBI path / IPC error so
|
||||
* callers can fall back to the regular foliate-js
|
||||
* `DocumentLoader.open()` pipeline with no behavioural change.
|
||||
*
|
||||
* `readMobiMetadata` short-circuits MOBI.open()'s expensive init()
|
||||
* (which walks every text record), keeping import roughly as fast
|
||||
* as the previous stub-style path while ensuring `Book.metadata`
|
||||
* matches the reader path exactly — including
|
||||
* `metadata.identifier === mobi.uid.toString()`, the PalmDB UID
|
||||
* existing libraries' `metaHash` was computed against.
|
||||
*
|
||||
* `fileobj` must be the same `File` the importer plans to keep using
|
||||
* for the rest of `importBook`; we route it directly into foliate so
|
||||
* we don't `openFile` twice (which can be expensive on mobile where
|
||||
* `Books` BaseDir routes through native scoped-storage).
|
||||
*/
|
||||
export const tryNativeParseMobi = async (
|
||||
filePath: string | undefined,
|
||||
fileobj: File,
|
||||
): Promise<NativeParsedMobi | null> => {
|
||||
if (!isEligibleMobiPath(filePath)) return null;
|
||||
try {
|
||||
const rust = await invoke<RustParsedMobi>('parse_mobi_metadata', { filePath });
|
||||
if (!rust || !rust.partialMd5) return null;
|
||||
|
||||
let coverBlob: Blob | null = null;
|
||||
if (rust.cover && rust.cover.bytes && rust.cover.mime) {
|
||||
const u8 =
|
||||
rust.cover.bytes instanceof Uint8Array
|
||||
? rust.cover.bytes
|
||||
: Uint8Array.from(rust.cover.bytes);
|
||||
if (u8.byteLength > 0) {
|
||||
// Slice into a fresh ArrayBuffer to satisfy lib.dom Blob typings
|
||||
// (which require BlobPart = ArrayBuffer/ArrayBufferView<ArrayBuffer>,
|
||||
// not the ArrayBufferLike that the Uint8Array constructor exposes).
|
||||
const ab = u8.buffer.slice(u8.byteOffset, u8.byteOffset + u8.byteLength) as ArrayBuffer;
|
||||
coverBlob = new Blob([ab], { type: rust.cover.mime });
|
||||
}
|
||||
}
|
||||
|
||||
const mobiModule = (await import('foliate-js/mobi.js')) as unknown as {
|
||||
readMobiMetadata: (
|
||||
file: File,
|
||||
opts?: { unzlib?: (buf: Uint8Array) => Uint8Array },
|
||||
) => Promise<{
|
||||
metadata: BookMetadata;
|
||||
getCover: () => Promise<Blob | null | undefined>;
|
||||
}>;
|
||||
};
|
||||
const fflate = (await import('foliate-js/vendor/fflate.js')) as unknown as {
|
||||
unzlibSync: (buf: Uint8Array) => Uint8Array;
|
||||
};
|
||||
const { metadata, getCover } = await mobiModule.readMobiMetadata(fileobj, {
|
||||
unzlib: fflate.unzlibSync,
|
||||
});
|
||||
|
||||
return {
|
||||
partialMd5: rust.partialMd5,
|
||||
format: inferMobiFormat(filePath),
|
||||
bookDoc: buildBookDocStub(metadata, coverBlob, getCover),
|
||||
};
|
||||
} catch (err) {
|
||||
console.warn('[tauriMobiBridge] native parse failed, falling back to JS:', err);
|
||||
return null;
|
||||
}
|
||||
};
|
||||
@@ -14,6 +14,28 @@ export default defineConfig({
|
||||
resolve: {
|
||||
conditions: ['development'],
|
||||
},
|
||||
optimizeDeps: {
|
||||
include: [
|
||||
'@supabase/supabase-js',
|
||||
'@tauri-apps/plugin-fs',
|
||||
'@tauri-apps/plugin-http',
|
||||
'@tauri-apps/api/path',
|
||||
'@tauri-apps/api/core',
|
||||
'@zip.js/zip.js',
|
||||
'franc-min',
|
||||
'iso-639-2',
|
||||
'iso-639-3',
|
||||
'js-md5',
|
||||
'jwt-decode',
|
||||
'uuid',
|
||||
],
|
||||
exclude: [
|
||||
'@pdfjs/pdf.min.mjs',
|
||||
'@readest/turso-database-wasm',
|
||||
'@readest/turso-database-wasm-common',
|
||||
'@readest/turso-database-common',
|
||||
],
|
||||
},
|
||||
test: {
|
||||
include: ['src/**/*.tauri.test.ts'],
|
||||
setupFiles: ['./vitest.tauri.setup.ts'],
|
||||
|
||||
+1
-1
Submodule packages/foliate-js updated: 70d77aa747...91191cafc4
Reference in New Issue
Block a user