Files
readest/apps/readest-app/src/utils/tauriEpubBridge.ts
T
loveheaven 11d796361e perf(import+open): native Rust EPUB/MOBI parser, OPF prefetch, parallel TOC enrichment (#4369)
* perf(epub): add native EPUB parser in Rust

Introduce a Rust-side EPUB pre-parser exposing three Tauri commands:

  * parse_epub_metadata     - title/author/cover + partialMD5 in one
                              shot, for the import hot path
  * parse_epub_full         - OPF + nav.xhtml + toc.ncx bytes plus a
                              manifest size table, for the reader open
                              hot path
  * extract_epub_cover_full - full-resolution cover bytes, for the
                              lock-screen wallpaper writer

All three avoid ferrying multi-MB blobs across the JS<->Rust IPC
boundary. Cover bytes returned by parse_epub_metadata are downscaled
to a webview-friendly JPEG when the long edge exceeds the library
thumbnail size.

No JS callers yet -- wired up in the following commits.

* perf(import): use native EPUB parser and downscale covers on Tauri targets

On Tauri (desktop/iOS/Android), importBook now forwards EPUB
metadata + cover extraction to the Rust parse_epub_metadata
command and reuses the partialMD5 it returns, skipping the
foliate-js full archive parse and the second pass over the file
for hashing.

As a side effect, the cover written to cover.png is downscaled
to a webview-friendly JPEG (long edge <= 512px), shrinking the
on-disk thumbnail from multi-MB to ~30-60KB per book. To keep
the lock-screen wallpaper feature unchanged, useAutoSaveBookCover
now pulls the original full-resolution cover via the Rust
extract_epub_cover_full command instead of copying the (now
downscaled) cover.png; falls back to the thumbnail when the
native path is unavailable.

Web targets and non-EPUB formats keep the existing path.

* perf(reader): prefetch EPUB OPF/nav from Rust on book open

When opening an EPUB on Tauri targets, DocumentLoader now calls the
Rust parse_epub_full command up-front to pull the OPF, EPUB3 nav,
NCX and the central-directory size map in a single IPC. The
foliate-js zip loader is wrapped so that loadText() of these
entries (and a synthetic META-INF/container.xml) is served from
that in-memory cache without inflating through zip.js, while
all other assets keep flowing through the original loader.

A small in-flight dedupe is added to the spine-text loader so the
nav pipeline (loadText + createDocument back-to-back on the same
href) doesn't pay for two zip.js inflate calls per chapter on
first open.

Reader store / app service plumbing: readerStore.openBook now
resolves an absolute on-disk path via the new
appService.resolveNativeBookFilePath / bookService.resolveNativeBookFilePath
helper and threads it into DocumentLoader as nativeFilePath so
the prefetch can fire. Web targets, non-EPUB formats and books
without a managed/external on-disk path skip the prefetch and
take the original code path.

* perf(nav): parallelize section scans and memoize fragment lookups

computeBookNav now processes sections via Promise.all instead of
a sequential for-loop, and within each section issues loadText()
and createDocument() concurrently. Combined with the in-flight
loadText dedupe added to the zip loader, each chapter pays for a
single zip inflate per nav build, and the inflates of different
chapters overlap.

enrichTocFromNavElements is restructured into two concurrent
phases: a cheap '<nav' substring filter on the inflated text, and
a parsed-document walk for the survivors. Most chapters fall out
in phase 1 without ever being parsed.

In fragments.ts, calculateFragmentSize now consults a
per-section position cache (makeFragmentPositionCache) so the
N-fragment loop is O(N) over the chapter HTML instead of O(N²).
A small isCfiAddressable guard is added to skip elements that
foliate-js's CFI generator can't address (documentElement, body
itself, detached nodes, nodes outside <body>) — these previously
threw and spammed console.warn for every fragment, now they
silently fall back to the section CFI.

* perf(import): use native MOBI/AZW/AZW3 parser on Tauri targets

On Tauri (desktop/iOS/Android), importBook now forwards
MOBI/AZW/AZW3/PRC metadata + cover extraction to the Rust
parse_mobi_metadata command and reuses the partialMD5 it returns,
skipping the foliate-js full-buffer parse and the second pass over
the file for hashing. Mirrors the existing EPUB native fast-path
added in e3fc4767 — bookService tries EPUB first, then MOBI; both
bridges fall back to the foliate-js DocumentLoader when the native
path is unavailable (web target, parse error, format mismatch).

The new mobi_parser is built on the mobi crate (KF7+KF8 reader,
zero JS-side touch). It reads title, author, publisher, ISBN, ASIN,
publish date, language, subjects and description from the MobiHeader
+ EXTH records, resolves the EXTH 201 cover offset against the PDB
image-record table (with ThumbOffset / first-image fallbacks), and
strips KindleGen's HTML wrapping in EXTH 103 so the description goes
into the library DB as plain text. The parsed cover is funneled
through the same maybe_resize_cover path as EPUB, so MOBI library
thumbnails are also clamped to a 512px-long-edge JPEG.

Cover-resize / partialMD5 / RawCoverImage are extracted into a new
parser_common module shared between epub_parser and mobi_parser, so
a single tweak (e.g. raising the thumbnail target) applies to every
native importer and the partialMD5 implementation can't drift between
the two paths (a divergent algorithm would silently re-import every
existing book under a new hash on the first run).

Web targets and non-Kindle formats keep the existing path.

* test(tauri): verify native Rust EPUB parser parity with foliate-js

Add a Tauri WebView parity suite (epub-parser-parity.tauri.test.ts) that
cross-checks the native Rust parser against foliate-js on the same fixtures:
parse_epub_metadata / parse_epub_full (title, author, language, identifier,
publisher, published, subjects, partialMD5, OPF + per-entry size table), and
that opening with the native prefetch produces the same BookDoc and
computeBookNav (TOC) output as the pure-JS path.

Fix a parity divergence the suite caught: the Rust OPF parser mapped
dcterms:modified onto `published`, but foliate-js keeps them separate and
leaves `published` empty -- so EPUB3 books carrying only the mandatory
dcterms:modified got a bogus publication date on the native import path. Map
only dc:date now; add regression tests.

Test infra:
- vitest.tauri.config.mts: add optimizeDeps (mirroring vitest.browser.config)
  so foliate-js-importing tauri tests load -- otherwise esbuild's dep scan
  can't resolve '@pdfjs/pdf.min.mjs', pre-bundling is skipped, and the CJS
  deps fail to import ("Importing a module script failed").
- capabilities-extra/webdriver.json: fix __test__ -> __tests__ fs scope typo
  so import tests can open fixtures under src/__tests__/.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(import): foliate-js owns EPUB/MOBI metadata via standalone extractors

Rust contributes only the mechanical work that's expensive on a
WebView — partialMD5, the downscaled cover, and (for EPUB) the raw
OPF bytes Rust already had to read for cover resolution. Metadata
extraction is delegated to foliate-js's two new standalone entry
points (`parseEpubMetadataFromXML`, `readMobiMetadata`) so the
import-path BookDoc and the reader-path BookDoc share a single
parser implementation.

EPUB
- `parse_epub_metadata` returns
  `{ partialMd5, cover, coverMime, opfPath, opfBytes }`. OPF bytes
  are a free byproduct of the cover-resolution scan.
- `tryNativeParseEpub` runs `parseEpubMetadataFromXML` on the OPF
  bytes and assembles a lightweight BookDoc stub (metadata +
  getCover). The importer doesn't drive `DocumentLoader.open()`, so
  no zip central-directory scan, no nav/ncx inflate, no spine walk.
- `coverMime` is preserved so `bookService.importBook`'s
  `cover.type === 'image/svg+xml'` branch still routes SVG covers
  through svg2png.

MOBI / AZW / AZW3 / PRC
- `parse_mobi_metadata` returns `{ partialMd5, cover, coverMime }`.
  `tryNativeParseMobi` runs foliate's `readMobiMetadata` on the
  same File, which uses `MOBI.open(file, { metadataOnly: true })`
  to parse PalmDB + MobiHeader + EXTH and short-circuit before the
  MOBI6 / KF8 init() that walks every text record.
- `Book.metadata.identifier` is foliate's `mobi.uid.toString()`
  (PalmDB UID), the canonical MOBI identifier the reader path uses.

bookService.importBook
- EPUB and MOBI native branches consume the bridge's BookDoc stub
  directly. The stub's `getCover()` returns the Rust-downscaled
  blob, falling back to foliate's own `getCover` thunk when Rust
  didn't extract a cover.

Other
- Drop the unused `base64` Rust dependency: cover bytes go over IPC
  as `Vec<u8>` (Tauri 2 transports them natively, like opfBytes /
  navBytes / ncxBytes).
- Drop the `nativePrefetch` option on `DocumentLoaderOptions`; no
  caller passes it. `nativeFilePath` keeps driving `parse_epub_full`
  on the open hot path.

Tests
- vitest.tauri parity test asserts byte-equal partialMD5, cover
  presence parity, OPF bytes that decode to a real `<package>`
  document, and that `parseEpubMetadataFromXML` on those bytes
  produces the same user-visible metadata fields (title / author /
  language / identifier / published) as `DocumentLoader.open()`.

* test(tauri): add War and Peace MOBI fixture for native parser parity

The .tauri parser-parity suite previously had no .mobi/.azw3 asset, so the native MOBI parser (metadata + EXTH cover resolution) was uncovered. Adds a real KF8 MOBI ("War and Peace") to enable MOBI parity coverage against foliate-js.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore(foliate-js): bump submodule to readest/foliate-js main (91191ca)

Replaces the ad-hoc 02f435a with the merged main commit 91191ca, which lands the standalone OPF/MOBI metadata extractors (parseEpubMetadataFromXML, readMobiMetadata) the import fast-path depends on (foliate#19), plus the RTL multi-view rect-mapper fix (foliate#20). The extractor code is byte-identical to 02f435a, so the bridges are unaffected.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Huang Xin <chrox.huang@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-10 16:58:25 +02:00

256 lines
11 KiB
TypeScript

// JS<->Rust EPUB bridge for Tauri targets.
//
// Architectural split:
//
// * Rust handles the *mechanical* zip work that's expensive on a
// WebView: opening the zip + central-directory parse, partialMD5
// over the file, locating the cover entry, decoding/resizing the
// cover image, and (on the open hot path) prefetching nav/ncx
// bytes + the entry-size map. It also forwards the OPF bytes it
// already had to read for cover resolution.
// * foliate-js stays the single source of truth for OPF metadata
// extraction (title, author, identifier, language map, refines /
// `belongs-to-collection` graph, ONIX5 codelists, …). The import
// path runs foliate's `parseEpubMetadataFromXML` directly on the
// OPF bytes Rust hands over — no zip.js central-directory scan,
// no nav/ncx inflate, no spine traversal — keeping the import
// hot path fast while ensuring `Book.metadata` stays byte-stable
// against what the reader path produces.
//
// Two Tauri commands back this:
//
// * parse_epub_metadata — import path. Returns
// `{ partialMd5, cover, coverMime, opfPath,
// opfBytes }`. The bridge runs foliate's
// OPF-only metadata extractor on
// `opfBytes` and assembles a lightweight
// BookDoc stub the importer consumes.
// * parse_epub_full — open path. Returns OPF prefetch + nav/
// ncx bytes + entry-size map for the
// DocumentLoader on the reader hot path.
//
// Avoids ferrying multi-MB blobs across the JS<->Rust IPC boundary
// and is a no-op on the web platform.
import { invoke } from '@tauri-apps/api/core';
import { isTauriAppPlatform } from '@/services/environment';
import type { BookDoc, BookMetadata } from '@/libs/document';
// ─── shared helpers ──────────────────────────────────────────────────
const isEligibleEpubPath = (filePath: string | undefined): filePath is string =>
!!filePath && isTauriAppPlatform() && /\.epub$/i.test(filePath);
/**
* Convert a `Vec<u8>` returned by Rust over Tauri's IPC into a plain
* `Uint8Array`. Depending on Tauri's serializer / WebView the wire form
* is either a `number[]` or already a typed array — normalize both.
*/
const toUint8Array = (bytes: number[] | Uint8Array): Uint8Array =>
bytes instanceof Uint8Array ? bytes : new Uint8Array(bytes);
/** Decode a UTF-8 byte buffer (Rust `Vec<u8>` over IPC) into a string. */
const bytesArrayToString = (bytes: number[] | Uint8Array): string =>
new TextDecoder('utf-8').decode(toUint8Array(bytes));
// ─── parse_epub_metadata (import path) ───────────────────────────────
interface RustParsedEpubMetadata {
/** partialMD5 of the EPUB file. Same algorithm as utils/md5.ts::partialMD5. */
partialMd5: string;
/** Pre-resized cover bytes (Vec<u8> over IPC), or null/absent when the
* EPUB has no cover. The Rust side runs the resize + JPEG re-encode
* through the `image` crate, which is materially faster than a
* `createImageBitmap` + canvas round-trip on Android mid-tier devices
* during bulk imports. */
cover?: number[] | Uint8Array | null;
/** MIME of `cover` after the (optional) re-encode. Always paired with
* `cover`. We propagate it so the JS-side SVG-cover branch in
* `bookService.importBook` (which checks `cover.type === "image/svg+xml"`
* to route through svg2png) keeps working even on the native fast path. */
coverMime?: string | null;
/** OPF zip path. Always present when `partialMd5` is. */
opfPath: string;
/** OPF bytes — Rust read these for cover resolution; we forward them
* so the importer can run foliate's OPF metadata extractor without a
* second zip access. */
opfBytes: number[] | Uint8Array;
}
export interface NativeParsedEpub {
/** partialMD5 of the file, ready to use as the `Book.hash`. */
partialMd5: string;
/** Lightweight BookDoc stub: only `metadata` and `getCover()` are
* populated, which is all `bookService.importBook` consults on the
* import hot path. Sections / TOC / fixed-layout detection are
* populated lazily by the reader when the user actually opens the
* book (which goes through the regular `DocumentLoader` path). */
bookDoc: BookDoc;
}
/**
* Build a BookDoc stub for the importer.
*
* `metadata` is whatever foliate-js's `parseEpubMetadataFromXML` returns
* for the OPF Rust handed us — byte-stable against the reader path.
* `getCover()` returns the Rust-downscaled blob; everything else is the
* minimum the importer reads (it never touches `sections` / `toc` /
* `splitTOCHref`).
*/
const buildBookDocStub = (metadata: BookMetadata, coverBlob: Blob | null): BookDoc => {
const stub = {
metadata,
rendition: {},
dir: 'ltr',
toc: [],
sections: [],
splitTOCHref: () => [null, null],
getCover: async () => coverBlob,
} as unknown as BookDoc;
return stub;
};
/**
* Try the import-time native fast-path. Returns `null` on web platform,
* non-EPUB path, or any IPC / parse error so callers can fall back to
* the regular foliate-js `DocumentLoader.open()` pipeline with no
* behavioural change.
*
* On success the bridge:
* 1. invokes the Rust `parse_epub_metadata` command (one IPC, one
* zip open, one partialMD5 pass);
* 2. parses the returned OPF bytes through foliate-js's exported
* `parseEpubMetadataFromXML`, so the resulting `metadata` shape
* (refines chains, ONIX5 codelists, language maps,
* `belongs-to-collection`, …) matches the reader path exactly;
* 3. wraps the cover bytes into a Blob carrying the Rust-supplied
* MIME (so `bookService.importBook`'s `cover.type === 'image/svg+xml'`
* branch can still route through svg2png).
*/
export const tryNativeParseEpub = async (
filePath: string | undefined,
): Promise<NativeParsedEpub | null> => {
if (!isEligibleEpubPath(filePath)) return null;
try {
const rust = await invoke<RustParsedEpubMetadata>('parse_epub_metadata', {
filePath,
});
if (!rust || !rust.partialMd5 || !rust.opfPath || !rust.opfBytes) return null;
// foliate-js exposes `parseEpubMetadataFromXML` so callers that
// already have OPF bytes (us — Rust just read them out of the zip)
// can derive `Book.metadata` without driving the full `EPUB.init()`
// (which would force `@zip.js/zip.js` to scan the central directory
// and inflate nav/ncx files the importer never reads). Dynamic
// import keeps the bridge tree-shakable on web builds where this
// path is never reached.
const epubModule = (await import('foliate-js/epub.js')) as unknown as {
parseEpubMetadataFromXML: (xml: string) => { metadata: BookMetadata };
};
const opfXml = bytesArrayToString(rust.opfBytes);
const { metadata } = epubModule.parseEpubMetadataFromXML(opfXml);
let coverBlob: Blob | null = null;
if (rust.cover && rust.coverMime) {
const bytes = toUint8Array(rust.cover);
if (bytes.byteLength > 0) {
// Slice into a fresh ArrayBuffer to satisfy lib.dom Blob typings
// (which require BlobPart = ArrayBuffer/ArrayBufferView<ArrayBuffer>,
// not the ArrayBufferLike that the Uint8Array constructor exposes).
const ab = bytes.buffer.slice(
bytes.byteOffset,
bytes.byteOffset + bytes.byteLength,
) as ArrayBuffer;
coverBlob = new Blob([ab], { type: rust.coverMime });
}
}
return {
partialMd5: rust.partialMd5,
bookDoc: buildBookDocStub(metadata, coverBlob),
};
} catch (err) {
console.warn('[tauriEpubBridge] native parse failed, falling back to JS:', err);
return null;
}
};
// ─── parse_epub_full (open hot-path prefetch) ────────────────────────
interface RustParsedEpubFull {
partialMd5: string;
opfPath: string;
opfBytes: number[] | Uint8Array;
navPath?: string | null;
navBytes?: number[] | Uint8Array | null;
ncxPath?: string | null;
ncxBytes?: number[] | Uint8Array | null;
/**
* Map: zip entry name → uncompressed size in bytes. Sent over IPC as a
* plain object (`{ "OEBPS/x.html": 12345, ... }`) and rehydrated into a
* Map below for O(1) `getSize()` calls.
*/
sizes: Record<string, number>;
}
export interface NativeEpubPrefetch {
/**
* Map of zip-path → text content. Populated for the OPF, EPUB3 nav doc,
* NCX (if present), and a synthetic META-INF/container.xml that points
* foliate-js at our OPF path. Anything not in the map falls through to
* the regular zip.js loadText path.
*/
textCache: Map<string, string>;
/** Map of zip-path → uncompressed byte size, for foliate-js getSize(). */
sizes: Map<string, number>;
/** partialMD5 of the file, returned alongside the prefetch in case the
* caller wants to reuse it (e.g. to set Book.hash without rehashing). */
partialMd5: string;
}
/**
* Build the minimal META-INF/container.xml that foliate-js's EPUB.init()
* looks at to find the OPF. We synthesize this from `opfPath` so the JS
* side never has to inflate the real container entry from the zip.
*/
const buildContainerXml = (opfPath: string): string =>
`<?xml version="1.0" encoding="UTF-8"?>` +
`<container version="1.0" xmlns="urn:oasis:names:tc:opendocument:xmlns:container">` +
`<rootfiles>` +
`<rootfile full-path="${opfPath}" media-type="application/oebps-package+xml"/>` +
`</rootfiles>` +
`</container>`;
/**
* Try to prefetch OPF/nav/ncx + entry sizes for an EPUB via Rust on the
* reader open hot path. Returns null when the native path is unavailable
* (web platform, missing file path, IPC error) so the caller can fall
* back to the regular zip.js-only DocumentLoader.
*/
export const tryNativePrefetchEpub = async (
filePath: string | undefined,
): Promise<NativeEpubPrefetch | null> => {
if (!isEligibleEpubPath(filePath)) return null;
try {
const rust = await invoke<RustParsedEpubFull>('parse_epub_full', {
filePath,
});
if (!rust || !rust.partialMd5 || !rust.opfPath || !rust.opfBytes) return null;
const textCache = new Map<string, string>();
textCache.set('META-INF/container.xml', buildContainerXml(rust.opfPath));
textCache.set(rust.opfPath, bytesArrayToString(rust.opfBytes));
if (rust.navPath && rust.navBytes) {
textCache.set(rust.navPath, bytesArrayToString(rust.navBytes));
}
if (rust.ncxPath && rust.ncxBytes) {
textCache.set(rust.ncxPath, bytesArrayToString(rust.ncxBytes));
}
const sizes = new Map<string, number>(Object.entries(rust.sizes ?? {}));
return { textCache, sizes, partialMd5: rust.partialMd5 };
} catch (err) {
console.warn('[tauriEpubBridge] native prefetch failed, falling back to JS:', err);
return null;
}
};