11d796361e
* perf(epub): add native EPUB parser in Rust
Introduce a Rust-side EPUB pre-parser exposing three Tauri commands:
* parse_epub_metadata - title/author/cover + partialMD5 in one
shot, for the import hot path
* parse_epub_full - OPF + nav.xhtml + toc.ncx bytes plus a
manifest size table, for the reader open
hot path
* extract_epub_cover_full - full-resolution cover bytes, for the
lock-screen wallpaper writer
All three avoid ferrying multi-MB blobs across the JS<->Rust IPC
boundary. Cover bytes returned by parse_epub_metadata are downscaled
to a webview-friendly JPEG when the long edge exceeds the library
thumbnail size.
No JS callers yet -- wired up in the following commits.
* perf(import): use native EPUB parser and downscale covers on Tauri targets
On Tauri (desktop/iOS/Android), importBook now forwards EPUB
metadata + cover extraction to the Rust parse_epub_metadata
command and reuses the partialMD5 it returns, skipping the
foliate-js full archive parse and the second pass over the file
for hashing.
As a side effect, the cover written to cover.png is downscaled
to a webview-friendly JPEG (long edge <= 512px), shrinking the
on-disk thumbnail from multi-MB to ~30-60KB per book. To keep
the lock-screen wallpaper feature unchanged, useAutoSaveBookCover
now pulls the original full-resolution cover via the Rust
extract_epub_cover_full command instead of copying the (now
downscaled) cover.png; falls back to the thumbnail when the
native path is unavailable.
Web targets and non-EPUB formats keep the existing path.
* perf(reader): prefetch EPUB OPF/nav from Rust on book open
When opening an EPUB on Tauri targets, DocumentLoader now calls the
Rust parse_epub_full command up-front to pull the OPF, EPUB3 nav,
NCX and the central-directory size map in a single IPC. The
foliate-js zip loader is wrapped so that loadText() of these
entries (and a synthetic META-INF/container.xml) is served from
that in-memory cache without inflating through zip.js, while
all other assets keep flowing through the original loader.
A small in-flight dedupe is added to the spine-text loader so the
nav pipeline (loadText + createDocument back-to-back on the same
href) doesn't pay for two zip.js inflate calls per chapter on
first open.
Reader store / app service plumbing: readerStore.openBook now
resolves an absolute on-disk path via the new
appService.resolveNativeBookFilePath / bookService.resolveNativeBookFilePath
helper and threads it into DocumentLoader as nativeFilePath so
the prefetch can fire. Web targets, non-EPUB formats and books
without a managed/external on-disk path skip the prefetch and
take the original code path.
* perf(nav): parallelize section scans and memoize fragment lookups
computeBookNav now processes sections via Promise.all instead of
a sequential for-loop, and within each section issues loadText()
and createDocument() concurrently. Combined with the in-flight
loadText dedupe added to the zip loader, each chapter pays for a
single zip inflate per nav build, and the inflates of different
chapters overlap.
enrichTocFromNavElements is restructured into two concurrent
phases: a cheap '<nav' substring filter on the inflated text, and
a parsed-document walk for the survivors. Most chapters fall out
in phase 1 without ever being parsed.
In fragments.ts, calculateFragmentSize now consults a
per-section position cache (makeFragmentPositionCache) so the
N-fragment loop is O(N) over the chapter HTML instead of O(N²).
A small isCfiAddressable guard is added to skip elements that
foliate-js's CFI generator can't address (documentElement, body
itself, detached nodes, nodes outside <body>) — these previously
threw and spammed console.warn for every fragment, now they
silently fall back to the section CFI.
* perf(import): use native MOBI/AZW/AZW3 parser on Tauri targets
On Tauri (desktop/iOS/Android), importBook now forwards
MOBI/AZW/AZW3/PRC metadata + cover extraction to the Rust
parse_mobi_metadata command and reuses the partialMD5 it returns,
skipping the foliate-js full-buffer parse and the second pass over
the file for hashing. Mirrors the existing EPUB native fast-path
added in e3fc4767 — bookService tries EPUB first, then MOBI; both
bridges fall back to the foliate-js DocumentLoader when the native
path is unavailable (web target, parse error, format mismatch).
The new mobi_parser is built on the mobi crate (KF7+KF8 reader,
zero JS-side touch). It reads title, author, publisher, ISBN, ASIN,
publish date, language, subjects and description from the MobiHeader
+ EXTH records, resolves the EXTH 201 cover offset against the PDB
image-record table (with ThumbOffset / first-image fallbacks), and
strips KindleGen's HTML wrapping in EXTH 103 so the description goes
into the library DB as plain text. The parsed cover is funneled
through the same maybe_resize_cover path as EPUB, so MOBI library
thumbnails are also clamped to a 512px-long-edge JPEG.
Cover-resize / partialMD5 / RawCoverImage are extracted into a new
parser_common module shared between epub_parser and mobi_parser, so
a single tweak (e.g. raising the thumbnail target) applies to every
native importer and the partialMD5 implementation can't drift between
the two paths (a divergent algorithm would silently re-import every
existing book under a new hash on the first run).
Web targets and non-Kindle formats keep the existing path.
* test(tauri): verify native Rust EPUB parser parity with foliate-js
Add a Tauri WebView parity suite (epub-parser-parity.tauri.test.ts) that
cross-checks the native Rust parser against foliate-js on the same fixtures:
parse_epub_metadata / parse_epub_full (title, author, language, identifier,
publisher, published, subjects, partialMD5, OPF + per-entry size table), and
that opening with the native prefetch produces the same BookDoc and
computeBookNav (TOC) output as the pure-JS path.
Fix a parity divergence the suite caught: the Rust OPF parser mapped
dcterms:modified onto `published`, but foliate-js keeps them separate and
leaves `published` empty -- so EPUB3 books carrying only the mandatory
dcterms:modified got a bogus publication date on the native import path. Map
only dc:date now; add regression tests.
Test infra:
- vitest.tauri.config.mts: add optimizeDeps (mirroring vitest.browser.config)
so foliate-js-importing tauri tests load -- otherwise esbuild's dep scan
can't resolve '@pdfjs/pdf.min.mjs', pre-bundling is skipped, and the CJS
deps fail to import ("Importing a module script failed").
- capabilities-extra/webdriver.json: fix __test__ -> __tests__ fs scope typo
so import tests can open fixtures under src/__tests__/.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(import): foliate-js owns EPUB/MOBI metadata via standalone extractors
Rust contributes only the mechanical work that's expensive on a
WebView — partialMD5, the downscaled cover, and (for EPUB) the raw
OPF bytes Rust already had to read for cover resolution. Metadata
extraction is delegated to foliate-js's two new standalone entry
points (`parseEpubMetadataFromXML`, `readMobiMetadata`) so the
import-path BookDoc and the reader-path BookDoc share a single
parser implementation.
EPUB
- `parse_epub_metadata` returns
`{ partialMd5, cover, coverMime, opfPath, opfBytes }`. OPF bytes
are a free byproduct of the cover-resolution scan.
- `tryNativeParseEpub` runs `parseEpubMetadataFromXML` on the OPF
bytes and assembles a lightweight BookDoc stub (metadata +
getCover). The importer doesn't drive `DocumentLoader.open()`, so
no zip central-directory scan, no nav/ncx inflate, no spine walk.
- `coverMime` is preserved so `bookService.importBook`'s
`cover.type === 'image/svg+xml'` branch still routes SVG covers
through svg2png.
MOBI / AZW / AZW3 / PRC
- `parse_mobi_metadata` returns `{ partialMd5, cover, coverMime }`.
`tryNativeParseMobi` runs foliate's `readMobiMetadata` on the
same File, which uses `MOBI.open(file, { metadataOnly: true })`
to parse PalmDB + MobiHeader + EXTH and short-circuit before the
MOBI6 / KF8 init() that walks every text record.
- `Book.metadata.identifier` is foliate's `mobi.uid.toString()`
(PalmDB UID), the canonical MOBI identifier the reader path uses.
bookService.importBook
- EPUB and MOBI native branches consume the bridge's BookDoc stub
directly. The stub's `getCover()` returns the Rust-downscaled
blob, falling back to foliate's own `getCover` thunk when Rust
didn't extract a cover.
Other
- Drop the unused `base64` Rust dependency: cover bytes go over IPC
as `Vec<u8>` (Tauri 2 transports them natively, like opfBytes /
navBytes / ncxBytes).
- Drop the `nativePrefetch` option on `DocumentLoaderOptions`; no
caller passes it. `nativeFilePath` keeps driving `parse_epub_full`
on the open hot path.
Tests
- vitest.tauri parity test asserts byte-equal partialMD5, cover
presence parity, OPF bytes that decode to a real `<package>`
document, and that `parseEpubMetadataFromXML` on those bytes
produces the same user-visible metadata fields (title / author /
language / identifier / published) as `DocumentLoader.open()`.
* test(tauri): add War and Peace MOBI fixture for native parser parity
The .tauri parser-parity suite previously had no .mobi/.azw3 asset, so the native MOBI parser (metadata + EXTH cover resolution) was uncovered. Adds a real KF8 MOBI ("War and Peace") to enable MOBI parity coverage against foliate-js.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(foliate-js): bump submodule to readest/foliate-js main (91191ca)
Replaces the ad-hoc 02f435a with the merged main commit 91191ca, which lands the standalone OPF/MOBI metadata extractors (parseEpubMetadataFromXML, readMobiMetadata) the import fast-path depends on (foliate#19), plus the RTL multi-view rect-mapper fix (foliate#20). The extractor code is byte-identical to 02f435a, so the bridges are unaffected.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Huang Xin <chrox.huang@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
147 lines
6.1 KiB
Rust
147 lines
6.1 KiB
Rust
// Shared helpers for the native import fast-path.
|
|
//
|
|
// Both the EPUB parser (`epub_parser`) and the MOBI/AZW/AZW3 parser
|
|
// (`mobi_parser`) need to:
|
|
// - compute the same `partialMD5` over the input file as `utils/md5.ts`,
|
|
// so the on-disk `Books/<hash>/...` layout stays stable regardless of
|
|
// which parser produced the entry,
|
|
// - clamp oversized cover artwork to the library-grid thumbnail size,
|
|
// re-encoding as JPEG q85 when downscaling actually fires.
|
|
//
|
|
// Keeping these in a single module avoids drift between the two import
|
|
// paths (a divergent partialMD5 implementation would silently re-import
|
|
// every existing book under a new hash on the first run after a change).
|
|
//
|
|
// `RawCoverImage` is the IPC-shaped struct returned to JS as a byte array
|
|
// + MIME pair; the JS bridges (`tauriEpubBridge.ts`, `tauriMobiBridge.ts`)
|
|
// turn it back into a `Uint8Array` before persisting through the existing
|
|
// `Books/<hash>/cover.<ext>` path.
|
|
|
|
use image::{codecs::jpeg::JpegEncoder, imageops::FilterType, GenericImageView};
|
|
use md5::{Digest, Md5};
|
|
use serde::Serialize;
|
|
use std::fs::File;
|
|
use std::io::{Cursor, Read, Seek, SeekFrom};
|
|
use std::path::Path;
|
|
|
|
/// Cover thumbnail target. Sized for the library grid (~250-300px @2x)
|
|
/// and the reader-sidebar / detail-view rows (which are smaller still).
|
|
/// Anything whose long edge is already at or below this stays untouched —
|
|
/// no decode/re-encode, original bytes are kept verbatim. Anything larger
|
|
/// is downscaled with [`COVER_RESIZE_FILTER`] and re-encoded as JPEG q85.
|
|
pub const COVER_MAX_LONG_EDGE: u32 = 512;
|
|
pub const COVER_JPEG_QUALITY: u8 = 85;
|
|
|
|
/// Resampling filter used to downscale covers. We deliberately use
|
|
/// `Triangle` (4-tap bilinear-ish) instead of `Lanczos3` (36-tap): at the
|
|
/// 512px-thumbnail scale the visual difference is imperceptible, but
|
|
/// Triangle is ~5-8x faster on a debug build (and ~3-5x faster on release)
|
|
/// because it touches far fewer source pixels per output pixel. Cover
|
|
/// thumbnails are displayed at <=300px in the UI, so any sharpening
|
|
/// advantage Lanczos3 would have is moot.
|
|
pub const COVER_RESIZE_FILTER: FilterType = FilterType::Triangle;
|
|
|
|
#[derive(Debug, Serialize)]
|
|
#[serde(rename_all = "camelCase")]
|
|
pub struct RawCoverImage {
|
|
/// Raw image bytes (serde will encode this as a JS array; the JS side
|
|
/// converts it back to a Uint8Array before writing to disk).
|
|
pub bytes: Vec<u8>,
|
|
pub mime: String,
|
|
}
|
|
|
|
/// Decode `bytes`, and if the long edge exceeds [`COVER_MAX_LONG_EDGE`],
|
|
/// resize ([`COVER_RESIZE_FILTER`], aspect ratio preserved) and re-encode
|
|
/// as JPEG at [`COVER_JPEG_QUALITY`].
|
|
///
|
|
/// On any decode/encode failure we fall back to the original bytes + the
|
|
/// caller-provided MIME so a malformed (but viewable) cover still makes it
|
|
/// to disk. `hint_mime` is informative only — `image::load_from_memory`
|
|
/// sniffs the actual format from the magic bytes, so misclaimed MIMEs in
|
|
/// the source container don't trip us up.
|
|
pub fn maybe_resize_cover(bytes: Vec<u8>, hint_mime: &str) -> (Vec<u8>, String) {
|
|
let img = match image::load_from_memory(&bytes) {
|
|
Ok(i) => i,
|
|
Err(_) => return (bytes, hint_mime.to_string()),
|
|
};
|
|
let (w, h) = img.dimensions();
|
|
if w.max(h) <= COVER_MAX_LONG_EDGE {
|
|
return (bytes, hint_mime.to_string());
|
|
}
|
|
let resized = img.resize(
|
|
COVER_MAX_LONG_EDGE,
|
|
COVER_MAX_LONG_EDGE,
|
|
COVER_RESIZE_FILTER,
|
|
);
|
|
let rgb = resized.to_rgb8();
|
|
|
|
let mut out = Vec::with_capacity(64 * 1024);
|
|
{
|
|
let mut encoder = JpegEncoder::new_with_quality(Cursor::new(&mut out), COVER_JPEG_QUALITY);
|
|
if encoder
|
|
.encode(
|
|
rgb.as_raw(),
|
|
rgb.width(),
|
|
rgb.height(),
|
|
image::ExtendedColorType::Rgb8,
|
|
)
|
|
.is_err()
|
|
{
|
|
return (bytes, hint_mime.to_string());
|
|
}
|
|
}
|
|
(out, "image/jpeg".to_string())
|
|
}
|
|
|
|
/// Mirror of `utils/md5.ts::partialMD5`:
|
|
/// step = 1024, size = 1024
|
|
/// for i in -1..=10:
|
|
/// start = min(file.size, step << (2*i)) // JS 32-bit shift
|
|
/// end = min(start + size, file.size)
|
|
/// if start >= file.size: break
|
|
/// hash file[start..end]
|
|
///
|
|
/// JS bit-shift operands are masked to their low 5 bits, so `1024 << -2`
|
|
/// actually means `1024 << 30`, which is far larger than any reasonable
|
|
/// file. That makes the very first iteration (i = -1) immediately break
|
|
/// for files smaller than ~1 GiB, leaving the hasher empty -> md5 of "" =
|
|
/// d41d8cd9... We must reproduce that behaviour bit-for-bit so existing
|
|
/// on-disk hashes (Books/<hash>/...) keep matching.
|
|
pub fn compute_partial_md5(path: &Path) -> std::io::Result<String> {
|
|
const STEP: u32 = 1024;
|
|
const CHUNK: u64 = 1024;
|
|
|
|
let mut file = File::open(path)?;
|
|
let file_len = file.metadata()?.len();
|
|
|
|
let mut hasher = Md5::new();
|
|
let mut buf = vec![0u8; CHUNK as usize];
|
|
|
|
for i in -1i32..=10 {
|
|
// JS evaluates `step << (2*i)` as a 32-bit shift, where the operand is
|
|
// implicitly masked to its low 5 bits. So `1024 << -2` is the same as
|
|
// `1024 << 30`, which overflows i32 to 0 (the high bits are dropped).
|
|
// For i = 0..=4 the shift is 0..=8 and stays within i32; for i >= 5
|
|
// the result overflows to 0 again. We mirror that with wrapping_shl.
|
|
let shift_amount = ((2 * i) as u32) & 31;
|
|
let shifted = (STEP as i32).wrapping_shl(shift_amount);
|
|
// Negative i32 results coerce to 0 here. JS's Math.min would surface
|
|
// the negative value, but the subsequent `start >= file.size` check
|
|
// would skip the read; clamping to 0 gives the same observable
|
|
// hash for non-empty files while avoiding negative seek offsets.
|
|
let raw = shifted.max(0) as u64;
|
|
let start = std::cmp::min(file_len, raw);
|
|
if start >= file_len {
|
|
break;
|
|
}
|
|
let end = std::cmp::min(start + CHUNK, file_len);
|
|
let to_read = (end - start) as usize;
|
|
file.seek(SeekFrom::Start(start))?;
|
|
let slice = &mut buf[..to_read];
|
|
file.read_exact(slice)?;
|
|
hasher.update(&slice[..]);
|
|
}
|
|
|
|
Ok(format!("{:x}", hasher.finalize()))
|
|
}
|