11d796361e
* perf(epub): add native EPUB parser in Rust
Introduce a Rust-side EPUB pre-parser exposing three Tauri commands:
* parse_epub_metadata - title/author/cover + partialMD5 in one
shot, for the import hot path
* parse_epub_full - OPF + nav.xhtml + toc.ncx bytes plus a
manifest size table, for the reader open
hot path
* extract_epub_cover_full - full-resolution cover bytes, for the
lock-screen wallpaper writer
All three avoid ferrying multi-MB blobs across the JS<->Rust IPC
boundary. Cover bytes returned by parse_epub_metadata are downscaled
to a webview-friendly JPEG when the long edge exceeds the library
thumbnail size.
No JS callers yet -- wired up in the following commits.
* perf(import): use native EPUB parser and downscale covers on Tauri targets
On Tauri (desktop/iOS/Android), importBook now forwards EPUB
metadata + cover extraction to the Rust parse_epub_metadata
command and reuses the partialMD5 it returns, skipping the
foliate-js full archive parse and the second pass over the file
for hashing.
As a side effect, the cover written to cover.png is downscaled
to a webview-friendly JPEG (long edge <= 512px), shrinking the
on-disk thumbnail from multi-MB to ~30-60KB per book. To keep
the lock-screen wallpaper feature unchanged, useAutoSaveBookCover
now pulls the original full-resolution cover via the Rust
extract_epub_cover_full command instead of copying the (now
downscaled) cover.png; falls back to the thumbnail when the
native path is unavailable.
Web targets and non-EPUB formats keep the existing path.
* perf(reader): prefetch EPUB OPF/nav from Rust on book open
When opening an EPUB on Tauri targets, DocumentLoader now calls the
Rust parse_epub_full command up-front to pull the OPF, EPUB3 nav,
NCX and the central-directory size map in a single IPC. The
foliate-js zip loader is wrapped so that loadText() of these
entries (and a synthetic META-INF/container.xml) is served from
that in-memory cache without inflating through zip.js, while
all other assets keep flowing through the original loader.
A small in-flight dedupe is added to the spine-text loader so the
nav pipeline (loadText + createDocument back-to-back on the same
href) doesn't pay for two zip.js inflate calls per chapter on
first open.
Reader store / app service plumbing: readerStore.openBook now
resolves an absolute on-disk path via the new
appService.resolveNativeBookFilePath / bookService.resolveNativeBookFilePath
helper and threads it into DocumentLoader as nativeFilePath so
the prefetch can fire. Web targets, non-EPUB formats and books
without a managed/external on-disk path skip the prefetch and
take the original code path.
* perf(nav): parallelize section scans and memoize fragment lookups
computeBookNav now processes sections via Promise.all instead of
a sequential for-loop, and within each section issues loadText()
and createDocument() concurrently. Combined with the in-flight
loadText dedupe added to the zip loader, each chapter pays for a
single zip inflate per nav build, and the inflates of different
chapters overlap.
enrichTocFromNavElements is restructured into two concurrent
phases: a cheap '<nav' substring filter on the inflated text, and
a parsed-document walk for the survivors. Most chapters fall out
in phase 1 without ever being parsed.
In fragments.ts, calculateFragmentSize now consults a
per-section position cache (makeFragmentPositionCache) so the
N-fragment loop is O(N) over the chapter HTML instead of O(N²).
A small isCfiAddressable guard is added to skip elements that
foliate-js's CFI generator can't address (documentElement, body
itself, detached nodes, nodes outside <body>) — these previously
threw and spammed console.warn for every fragment, now they
silently fall back to the section CFI.
* perf(import): use native MOBI/AZW/AZW3 parser on Tauri targets
On Tauri (desktop/iOS/Android), importBook now forwards
MOBI/AZW/AZW3/PRC metadata + cover extraction to the Rust
parse_mobi_metadata command and reuses the partialMD5 it returns,
skipping the foliate-js full-buffer parse and the second pass over
the file for hashing. Mirrors the existing EPUB native fast-path
added in e3fc4767 — bookService tries EPUB first, then MOBI; both
bridges fall back to the foliate-js DocumentLoader when the native
path is unavailable (web target, parse error, format mismatch).
The new mobi_parser is built on the mobi crate (KF7+KF8 reader,
zero JS-side touch). It reads title, author, publisher, ISBN, ASIN,
publish date, language, subjects and description from the MobiHeader
+ EXTH records, resolves the EXTH 201 cover offset against the PDB
image-record table (with ThumbOffset / first-image fallbacks), and
strips KindleGen's HTML wrapping in EXTH 103 so the description goes
into the library DB as plain text. The parsed cover is funneled
through the same maybe_resize_cover path as EPUB, so MOBI library
thumbnails are also clamped to a 512px-long-edge JPEG.
Cover-resize / partialMD5 / RawCoverImage are extracted into a new
parser_common module shared between epub_parser and mobi_parser, so
a single tweak (e.g. raising the thumbnail target) applies to every
native importer and the partialMD5 implementation can't drift between
the two paths (a divergent algorithm would silently re-import every
existing book under a new hash on the first run).
Web targets and non-Kindle formats keep the existing path.
* test(tauri): verify native Rust EPUB parser parity with foliate-js
Add a Tauri WebView parity suite (epub-parser-parity.tauri.test.ts) that
cross-checks the native Rust parser against foliate-js on the same fixtures:
parse_epub_metadata / parse_epub_full (title, author, language, identifier,
publisher, published, subjects, partialMD5, OPF + per-entry size table), and
that opening with the native prefetch produces the same BookDoc and
computeBookNav (TOC) output as the pure-JS path.
Fix a parity divergence the suite caught: the Rust OPF parser mapped
dcterms:modified onto `published`, but foliate-js keeps them separate and
leaves `published` empty -- so EPUB3 books carrying only the mandatory
dcterms:modified got a bogus publication date on the native import path. Map
only dc:date now; add regression tests.
Test infra:
- vitest.tauri.config.mts: add optimizeDeps (mirroring vitest.browser.config)
so foliate-js-importing tauri tests load -- otherwise esbuild's dep scan
can't resolve '@pdfjs/pdf.min.mjs', pre-bundling is skipped, and the CJS
deps fail to import ("Importing a module script failed").
- capabilities-extra/webdriver.json: fix __test__ -> __tests__ fs scope typo
so import tests can open fixtures under src/__tests__/.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(import): foliate-js owns EPUB/MOBI metadata via standalone extractors
Rust contributes only the mechanical work that's expensive on a
WebView — partialMD5, the downscaled cover, and (for EPUB) the raw
OPF bytes Rust already had to read for cover resolution. Metadata
extraction is delegated to foliate-js's two new standalone entry
points (`parseEpubMetadataFromXML`, `readMobiMetadata`) so the
import-path BookDoc and the reader-path BookDoc share a single
parser implementation.
EPUB
- `parse_epub_metadata` returns
`{ partialMd5, cover, coverMime, opfPath, opfBytes }`. OPF bytes
are a free byproduct of the cover-resolution scan.
- `tryNativeParseEpub` runs `parseEpubMetadataFromXML` on the OPF
bytes and assembles a lightweight BookDoc stub (metadata +
getCover). The importer doesn't drive `DocumentLoader.open()`, so
no zip central-directory scan, no nav/ncx inflate, no spine walk.
- `coverMime` is preserved so `bookService.importBook`'s
`cover.type === 'image/svg+xml'` branch still routes SVG covers
through svg2png.
MOBI / AZW / AZW3 / PRC
- `parse_mobi_metadata` returns `{ partialMd5, cover, coverMime }`.
`tryNativeParseMobi` runs foliate's `readMobiMetadata` on the
same File, which uses `MOBI.open(file, { metadataOnly: true })`
to parse PalmDB + MobiHeader + EXTH and short-circuit before the
MOBI6 / KF8 init() that walks every text record.
- `Book.metadata.identifier` is foliate's `mobi.uid.toString()`
(PalmDB UID), the canonical MOBI identifier the reader path uses.
bookService.importBook
- EPUB and MOBI native branches consume the bridge's BookDoc stub
directly. The stub's `getCover()` returns the Rust-downscaled
blob, falling back to foliate's own `getCover` thunk when Rust
didn't extract a cover.
Other
- Drop the unused `base64` Rust dependency: cover bytes go over IPC
as `Vec<u8>` (Tauri 2 transports them natively, like opfBytes /
navBytes / ncxBytes).
- Drop the `nativePrefetch` option on `DocumentLoaderOptions`; no
caller passes it. `nativeFilePath` keeps driving `parse_epub_full`
on the open hot path.
Tests
- vitest.tauri parity test asserts byte-equal partialMD5, cover
presence parity, OPF bytes that decode to a real `<package>`
document, and that `parseEpubMetadataFromXML` on those bytes
produces the same user-visible metadata fields (title / author /
language / identifier / published) as `DocumentLoader.open()`.
* test(tauri): add War and Peace MOBI fixture for native parser parity
The .tauri parser-parity suite previously had no .mobi/.azw3 asset, so the native MOBI parser (metadata + EXTH cover resolution) was uncovered. Adds a real KF8 MOBI ("War and Peace") to enable MOBI parity coverage against foliate-js.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(foliate-js): bump submodule to readest/foliate-js main (91191ca)
Replaces the ad-hoc 02f435a with the merged main commit 91191ca, which lands the standalone OPF/MOBI metadata extractors (parseEpubMetadataFromXML, readMobiMetadata) the import fast-path depends on (foliate#19), plus the RTL multi-view rect-mapper fix (foliate#20). The extractor code is byte-identical to 02f435a, so the bridges are unaffected.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Huang Xin <chrox.huang@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
247 lines
9.7 KiB
Rust
247 lines
9.7 KiB
Rust
// Native MOBI/AZW/AZW3 import path.
|
|
//
|
|
// Scope after PR review (mirrors `epub_parser`): this command no
|
|
// longer extracts MOBI metadata. The full set of EXTH semantics
|
|
// (placeholder filtering, ISBN-10/13 checksum validation, ASIN
|
|
// fallback, BCP-47 language normalisation, " & " / " and " / ";"
|
|
// author splitting, HTML-stripped descriptions, …) is left to
|
|
// foliate-js's `mobi.js`, which the JS side already runs through
|
|
// `DocumentLoader.open()` on the import path. Re-implementing those
|
|
// rules in Rust would silently diverge from the canonical parser
|
|
// used by the reader hot path; worse, switching `Book.metadata.identifier`
|
|
// from the foliate `mobi.uid` (PalmDB UID) to a Rust-derived ISBN/ASIN
|
|
// would change the metaHash for every existing MOBI in users' libraries
|
|
// and trigger spurious re-imports on upgrade.
|
|
//
|
|
// What `parse_mobi_metadata` still does on the import hot path:
|
|
// - compute partialMD5 over the file (matches `utils/md5.ts::partialMD5`,
|
|
// shared with `epub_parser` via `parser_common::compute_partial_md5`);
|
|
// - parse the PalmDB / MobiHeader / EXTH headers via the `mobi` crate
|
|
// just enough to locate the cover record;
|
|
// - locate the cover image (EXTH `CoverOffset` 201 → `ThumbOffset` 202
|
|
// → first image record fallback), sniff the MIME from magic bytes,
|
|
// and run `parser_common::maybe_resize_cover` to clamp it to the
|
|
// library-grid thumbnail target. Cover decode/resize stays here
|
|
// because the `image` crate is materially faster than the
|
|
// `createImageBitmap` + canvas round-trip on Android mid-tier
|
|
// devices, and bulk imports actually exercise that.
|
|
//
|
|
// Returned to JS via `parse_mobi_metadata`. The JS bridge wraps the
|
|
// downscaled bytes into a Blob and Proxies it onto foliate's BookDoc
|
|
// in place of the foliate `getCover()` (which would otherwise return
|
|
// the original-resolution bytes). foliate stays the source of truth
|
|
// for everything else.
|
|
|
|
use mobi::headers::ExthRecord;
|
|
use mobi::Mobi;
|
|
use serde::Serialize;
|
|
use std::path::Path;
|
|
|
|
use crate::parser_common::{compute_partial_md5, maybe_resize_cover, RawCoverImage};
|
|
|
|
#[derive(Debug, Serialize)]
|
|
#[serde(rename_all = "camelCase")]
|
|
pub struct ParsedMobi {
|
|
pub partial_md5: String,
|
|
/// `None` when the MOBI has no embedded cover image. Otherwise
|
|
/// the pre-resized bytes + sniffed MIME — the JS side wraps them
|
|
/// in a Blob and overrides foliate's `getCover()` so the on-disk
|
|
/// thumbnail wins over foliate's full-resolution decode.
|
|
pub cover: Option<RawCoverImage>,
|
|
}
|
|
|
|
/// Tauri command: parse a MOBI/AZW/AZW3 file's partialMD5 + cover and
|
|
/// return both in one IPC round-trip.
|
|
///
|
|
/// Runs on a blocking pool because `mobi::Mobi::from_path` reads the
|
|
/// whole file synchronously and parsing a 50 MB AZW3 can take tens of
|
|
/// milliseconds — long enough to want it off the Tauri main runtime.
|
|
#[tauri::command]
|
|
pub async fn parse_mobi_metadata(file_path: String) -> Result<ParsedMobi, String> {
|
|
tauri::async_runtime::spawn_blocking(move || parse_mobi_metadata_sync(&file_path))
|
|
.await
|
|
.map_err(|e| format!("join error: {e}"))?
|
|
}
|
|
|
|
fn parse_mobi_metadata_sync(file_path: &str) -> Result<ParsedMobi, String> {
|
|
let path = Path::new(file_path);
|
|
if !path.is_file() {
|
|
return Err(format!("file not found: {file_path}"));
|
|
}
|
|
|
|
let partial_md5 = compute_partial_md5(path).map_err(|e| format!("partial_md5 failed: {e}"))?;
|
|
|
|
let mobi = Mobi::from_path(path).map_err(|e| format!("parse mobi: {e}"))?;
|
|
|
|
let cover = extract_cover(&mobi).map(|raw| {
|
|
let (bytes, mime) = maybe_resize_cover(raw.bytes, &raw.mime);
|
|
RawCoverImage { bytes, mime }
|
|
});
|
|
|
|
Ok(ParsedMobi { partial_md5, cover })
|
|
}
|
|
|
|
/// Extract the *original* (un-resized) cover bytes from a MOBI / AZW / AZW3.
|
|
///
|
|
/// Mirrors `epub_parser::extract_epub_cover_full`: the import path stores a
|
|
/// downscaled thumbnail (via `maybe_resize_cover` inside
|
|
/// `parse_mobi_metadata_sync`) for the library grid, but features like the
|
|
/// Android / iOS lock-screen wallpaper want the full-resolution artwork. We
|
|
/// re-run the same EXTH lookup as the import path here, but skip the
|
|
/// downscale step and hand the raw record bytes back to JS along with a
|
|
/// MIME sniffed from the magic bytes.
|
|
///
|
|
/// Returns `Err` only when the file has no embedded cover at all.
|
|
#[tauri::command]
|
|
pub async fn extract_mobi_cover_full(file_path: String) -> Result<RawCoverImage, String> {
|
|
tauri::async_runtime::spawn_blocking(move || extract_mobi_cover_full_sync(&file_path))
|
|
.await
|
|
.map_err(|e| format!("join error: {e}"))?
|
|
}
|
|
|
|
fn extract_mobi_cover_full_sync(file_path: &str) -> Result<RawCoverImage, String> {
|
|
let path = Path::new(file_path);
|
|
if !path.is_file() {
|
|
return Err(format!("file not found: {file_path}"));
|
|
}
|
|
let mobi = Mobi::from_path(path).map_err(|e| format!("parse mobi: {e}"))?;
|
|
extract_cover(&mobi).ok_or_else(|| "no cover image in mobi".to_string())
|
|
}
|
|
|
|
/// Locate and decode the embedded cover image.
|
|
///
|
|
/// Strategy (mirrors what foliate-js's mobi.js does on the JS side):
|
|
/// 1. Read EXTH `CoverOffset` (record 201). The payload is a 4-byte
|
|
/// big-endian u32 giving an offset into the image-record subset.
|
|
/// Add `MobiHeader.first_image_index` to get a global PDB record
|
|
/// index, then look that record up in `Mobi::image_records()`.
|
|
/// 2. If 201 is missing, try `ThumbOffset` (record 202) the same way.
|
|
/// 3. If neither is present, fall back to the first image record —
|
|
/// MOBI generators almost always place the cover first, and a
|
|
/// "wrong but plausible" thumbnail is better than no thumbnail.
|
|
///
|
|
/// Returns `None` only when the file has no image records at all (rare
|
|
/// for real Kindle content).
|
|
fn extract_cover(mobi: &Mobi) -> Option<RawCoverImage> {
|
|
let images = mobi.image_records();
|
|
if images.is_empty() {
|
|
return None;
|
|
}
|
|
|
|
let first_image_index = mobi.metadata.mobi.first_image_index;
|
|
|
|
let exth_offset = read_exth_u32(mobi, ExthRecord::CoverOffset)
|
|
.or_else(|| read_exth_u32(mobi, ExthRecord::ThumbOffset));
|
|
|
|
let bytes: Vec<u8> = if let Some(off) = exth_offset {
|
|
// EXTH stores a *relative* offset; the absolute PDB record id
|
|
// is `first_image_index + off`. `image_records()` is filtered
|
|
// to image-only records, so we have to find the entry whose
|
|
// PdbRecord id matches the absolute id, not index linearly.
|
|
let target_id = first_image_index.saturating_add(off);
|
|
images
|
|
.iter()
|
|
.find(|r| r.record.id == target_id)
|
|
// Some files store the offset already pre-resolved into
|
|
// image_records()'s ordering; allow that as a fallback.
|
|
.or_else(|| images.get(off as usize))
|
|
.map(|r| r.content.to_vec())
|
|
.unwrap_or_else(|| images[0].content.to_vec())
|
|
} else {
|
|
images[0].content.to_vec()
|
|
};
|
|
|
|
if bytes.is_empty() {
|
|
return None;
|
|
}
|
|
|
|
let mime = sniff_image_mime(&bytes).to_string();
|
|
Some(RawCoverImage { bytes, mime })
|
|
}
|
|
|
|
/// Read the first occurrence of `record` and interpret its payload as
|
|
/// a 4-byte big-endian u32. EXTH offset records (201 / 202 / 116, etc.)
|
|
/// follow this convention. Returns `None` if the record is absent or
|
|
/// shorter than 4 bytes.
|
|
fn read_exth_u32(mobi: &Mobi, record: ExthRecord) -> Option<u32> {
|
|
let recs = mobi.metadata.exth.get_record(record)?;
|
|
let bytes = recs.first()?;
|
|
if bytes.len() < 4 {
|
|
return None;
|
|
}
|
|
Some(u32::from_be_bytes([bytes[0], bytes[1], bytes[2], bytes[3]]))
|
|
}
|
|
|
|
/// Best-effort MIME sniffing from magic bytes for the formats MOBI
|
|
/// covers are realistically stored as. Falls back to "image/jpeg" —
|
|
/// the dominant case — when the magic is unknown, because
|
|
/// `image::load_from_memory` (called downstream by
|
|
/// `maybe_resize_cover`) will detect the real format anyway and the
|
|
/// hint MIME is only used when we *don't* re-encode (small covers,
|
|
/// kept verbatim).
|
|
///
|
|
/// BMP is included because some early KindleGen builds (and a few
|
|
/// self-published .prc files) shipped BMP covers; the JS thumbnail
|
|
/// pipeline can render BMP via the same downscale path.
|
|
fn sniff_image_mime(bytes: &[u8]) -> &'static str {
|
|
if bytes.starts_with(&[0xFF, 0xD8, 0xFF]) {
|
|
"image/jpeg"
|
|
} else if bytes.starts_with(&[0x89, b'P', b'N', b'G', 0x0D, 0x0A, 0x1A, 0x0A]) {
|
|
"image/png"
|
|
} else if bytes.starts_with(b"GIF87a") || bytes.starts_with(b"GIF89a") {
|
|
"image/gif"
|
|
} else if bytes.starts_with(b"RIFF") && bytes.get(8..12) == Some(b"WEBP") {
|
|
"image/webp"
|
|
} else if bytes.starts_with(b"BM") {
|
|
"image/bmp"
|
|
} else {
|
|
"image/jpeg"
|
|
}
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
|
|
#[test]
|
|
fn sniff_image_mime_jpeg() {
|
|
assert_eq!(sniff_image_mime(&[0xFF, 0xD8, 0xFF, 0xE0]), "image/jpeg");
|
|
}
|
|
|
|
#[test]
|
|
fn sniff_image_mime_png() {
|
|
assert_eq!(
|
|
sniff_image_mime(&[0x89, b'P', b'N', b'G', 0x0D, 0x0A, 0x1A, 0x0A, 0, 0]),
|
|
"image/png"
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn sniff_image_mime_gif() {
|
|
assert_eq!(sniff_image_mime(b"GIF89a..."), "image/gif");
|
|
}
|
|
|
|
#[test]
|
|
fn sniff_image_mime_webp() {
|
|
let mut buf = Vec::new();
|
|
buf.extend_from_slice(b"RIFF");
|
|
buf.extend_from_slice(&[0, 0, 0, 0]);
|
|
buf.extend_from_slice(b"WEBP");
|
|
assert_eq!(sniff_image_mime(&buf), "image/webp");
|
|
}
|
|
|
|
#[test]
|
|
fn sniff_image_mime_unknown_falls_back_to_jpeg() {
|
|
assert_eq!(sniff_image_mime(&[0, 0, 0, 0]), "image/jpeg");
|
|
}
|
|
|
|
#[test]
|
|
fn sniff_image_mime_bmp() {
|
|
// BMP magic is "BM" followed by file size + reserved + offset.
|
|
let mut buf = Vec::new();
|
|
buf.extend_from_slice(b"BM");
|
|
buf.extend_from_slice(&[0; 12]);
|
|
assert_eq!(sniff_image_mime(&buf), "image/bmp");
|
|
}
|
|
}
|