75f1fafe9f229e4ef048a97b3857ae5c8b461699
125 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Huang Xin
|
ec45a080fc |
feat(metadata): surface calibre custom columns from EPUB metadata (#4939)
Parse calibre's embedded user metadata (custom columns) from the OPF in foliate-js, store it on BookMetadata.calibreColumns, render the columns in the book details view, and match column names and values in the library search so a value like a recommends tag can be found by typing it. Closes #4811 |
||
|
Huang Xin
|
42f9b8fe3c |
feat(tts): gapless Web Audio playback engine for Edge TTS with chapter timeline and seek (#4931)
* feat(tts): add PCM speech-bounds detection for sentence audio trimming Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(tts): add WSOLA time-stretch for pitch-preserved playback rate Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(tts): add sentence duration store with per-voice speaking-rate calibration Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(tts): serve edge audio as ArrayBuffer with in-flight fetch dedup Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(tts): add WebAudioPlayer with gapless chunk scheduling and backpressure Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(tts): play edge TTS through gapless Web Audio pipeline Replaces the per-sentence audio element with trimmed, time-stretched buffers scheduled on the shared AudioContext. Marks dispatch at audible time so schedule-ahead cannot run foliate's cursor past the voice; a decode failure or missing audio skips the chunk instead of wedging the session; pause and resume ride context suspend and resume with no iOS rewind hack; the object-URL cache is gone. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(tts): add section timeline with measured and estimated sentence durations Includes the foliate-js submodule bump for the getSentences export (fork branch feat/tts-get-sentences; fork PR must merge before this lands so the pinned SHA resolves). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(tts): expose section playback position and sentence-snapped seeking Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(tts): surface playback position and seek in the media session Position state is clamped, never skipped, so the lock-screen scrubber stays live when estimates overshoot; seekto units map per backend (native ms, web seconds). The AudioContext warms up in the tts-speak gesture path before any await, and the silent keep-alive element now runs on all platforms so desktop hardware media keys survive the removal of the per-sentence audio element. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(tts): add seekable chapter progress bar to the TTS panel The scrubber joins the transport cluster with a thin range-xs track and flanking tabular time labels so it cannot be misgrabbed for the chunky rate slider (which persists a global setting). States: reserved disabled slot until the lazy timeline lands, persists across chapter transitions, optimistic thumb with failure toast, monotonic position, tilde-prefixed estimated totals, sentence-event updates under e-ink. The popup grows only when a timeline-capable client is active. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: record deferred TTS listening-engine follow-ups Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: record background TTS decoupling design decisions Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(tts): slim the panel scrubber to a native track with remaining time Match the footer Jump to Location slider (plain native range: thin track, small thumb) instead of the chunky daisyUI pill, show remaining time with a minus prefix on the right, and drop the This chapter caption. Popup height shrinks accordingly. Verified live in Chrome. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: pin foliate-js to merged main with getSentences export Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(tts): catch autoplay rejection from the keep-alive element Running the silent keep-alive on all platforms exposed an un-awaited play() that headless Chromium rejects without a user gesture, failing CI on unhandled rejections while every test passed. The keep-alive is best-effort; the production path is gesture-qualified. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Huang Xin
|
4dbe9cc9f1 | fix: Sentry production hardening (release/OS tags, unhandled-rejection & render-loop guards) (#4929) | ||
|
Huang Xin
|
49391124c5 |
fix(reader): correct reading ruler direction for vertical-rl books (#4865) (#4879)
Vertical-rl (Japanese/Chinese vertical) books read top-to-bottom with columns progressing right-to-left, but getDirection only derived rtl from the horizontal dir/direction, which stays ltr for these books. As a result viewSettings.rtl was false and the reading ruler laid columns out left-to-right, advancing the band the wrong way (reverse reading order). Treat writing-mode: vertical-rl as RTL in getDirection so vertical-rl runs through the same rtl paths that horizontal-rtl already uses: the reading ruler coordinate mapping, page-turn tap mapping, footer navigation, and the progress bar. Page-turn taps for these books now follow the vertical-rl convention (tap left to go forward), matching horizontal-rtl behavior. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
24370ca511 |
feat(reader): render Markdown (.md) files at runtime (#774) (#4816)
Open standalone .md files in the reader without converting to EPUB. A new makeMarkdownBook (src/utils/md.ts) parses Markdown to sanitized HTML with marked + DOMPurify, splits the document into sections at H1 boundaries, and builds an in-memory foliate book (modeled on fb2.js) with a nested heading TOC. DocumentLoader routes .md/.markdown before the TXT path so a Markdown file served as text/plain is not converted to EPUB. Layout, font and theme settings apply the same as for any other format. Relative-image resolution and Markdown bundle/folder packages are left as follow-ups (a standalone file has no sibling-asset access on the web). Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
4fa7f76bc1 |
feat(payment): observability for store subscription webhooks (#4704)
* feat(payment): observability for store subscription webhooks Add monitoring for the App Store / Google Play webhooks so store-side subscription changes are observable on Cloudflare, where stored Workers Logs are head-sampled at 1% and would miss almost all low-volume webhook events. - Add iap/telemetry.ts: every webhook invocation emits a structured log line (streamed in full by `wrangler tail`) and a Cloudflare Analytics Engine data point (100% capture, independent of log sampling). Writes no-op off the Worker runtime, mirroring the getCloudflareContext guard in deepl/translate.ts. - Instrument both webhook routes to record outcome (handled, skipped, rejected, error), notification type, status, reason, and latency on every return path. - Add GET /api/cron/iap-reconcile: a CRON_SECRET-protected sweep that counts drift (rows still active while their store expiry has passed = a missed webhook) in both IAP tables and records a reconcile metric. Detection-only; never mutates state. - Add the IAP_WEBHOOK_AE Analytics Engine binding to wrangler.toml. New configuration: CRON_SECRET (reconcile auth) and an iap_webhooks Analytics Engine dataset bound as IAP_WEBHOOK_AE. The reconcile route is triggered on a schedule (a Cloudflare Cron Trigger worker that fetches the URL, or any external scheduler) with an Authorization bearer header. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(payment): move reconciliation to a dedicated cron Worker Replace the public CRON_SECRET-protected /api/cron/iap-reconcile route with a dedicated Cloudflare Cron Worker. A Cron Trigger invokes the worker's scheduled() handler directly, so there is no public HTTP surface and no shared request secret to manage - the strongest option on Cloudflare (OpenNext's generated worker only exports `fetch`, so the main worker cannot host a scheduled() handler). - Add workers/iap-reconcile: a self-contained worker (own package.json, tsconfig, wrangler.toml) matching the existing workers/send-email convention, registered in pnpm-workspace.yaml. Hourly Cron Trigger; reads the IAP tables via the Supabase service role and records a drift metric to the shared iap_webhooks Analytics Engine dataset. - Reconcile logic lives in workers/iap-reconcile/src/reconcile.ts and is unit-tested from the app suite. - Remove the public route and its test; drop the now-unused recordIapReconcile from iap/telemetry.ts (webhook telemetry unchanged). Configuration: set SUPABASE_URL and SUPABASE_SERVICE_ROLE_KEY as secrets on the worker and deploy it with `wrangler deploy` from its directory. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
359fdddcf4 |
feat(payment): handle App Store and Google Play subscription webhooks (#4701)
Add server-push endpoints so store-side subscription changes (cancel, refund, expire, renew, grace period) are reflected in the database, not only the in-app verification flow. Previously only Stripe had a webhook. - POST /api/apple/notifications: verify and decode App Store Server Notifications V2, resolve the user by original_transaction_id, map the notification type to a status, and update the subscription and plan. A single endpoint serves Sandbox and Production. Refunded one-time purchases are marked refunded and storage is recomputed. - POST /api/google/notifications: verify the Pub/Sub shared-secret token, decode the RTDN, resolve the user by purchase_token, re-verify against the Play Developer API (overriding the status for terminal events such as REVOKED/EXPIRED and grace period), and handle voided purchases. - Add an isEntitledStatus helper and reuse the existing createOrUpdateSubscription and plan-update logic shared with Stripe. New configuration: GOOGLE_RTDN_VERIFICATION_TOKEN (shared secret in the Pub/Sub push URL) and the optional GOOGLE_IAP_PACKAGE_NAME; Apple reuses APPLE_IAP_BUNDLE_ID and the existing service-account credentials. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
96d65d9960 |
feat(tts): add native local iOS TTS (AVSpeechSynthesizer) (#4697)
Implement on-device iOS text-to-speech using AVSpeechSynthesizer, mirroring the Android native TextToSpeech plugin so the shared NativeTTSClient drives both platforms through the same command and tts_events contract. - Swift NativeTTSPlugin: speak/stop/pause/resume/rate/pitch/voice and voice enumeration, with region-disambiguated duplicate voice names and a small preUtteranceDelay to avoid first-word clipping. - Enable the native TTS client on iOS in TTSController. - Make TTS teardown resilient: reset UI state up front and tear down the controller, media session, and background audio in parallel so a slow native shutdown can never leave the TTS icon or lock-screen session stuck on. - Keep iOS on navigator.mediaSession for the lock screen (Android uses the native foreground service), which restores the Edge TTS cover and current-sentence metadata. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
9e163fe746 |
fix(payment): reflect highest active plan across overlapping Stripe subscriptions (#4694)
* fix(payment): reflect highest active plan across overlapping Stripe subscriptions When a user upgrades Plus to Pro, both subscriptions stay active until the old one is cancelled. Each subscription webhook overwrote plans.plan with only that event's plan, so whichever webhook arrived last won and could downgrade the account back to plus. Derive plans.plan from the highest active (or trialing) subscription via a new getHighestActivePlan helper, used by createOrUpdateSubscription and by the cancellation handler so a still-active higher plan is preserved instead of dropping to free. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(payment): add opt-in live Stripe test for getHighestActivePlan Skipped by default (CI included); runs only when STRIPE_SECRET_KEY and STRIPE_TEST_CUSTOMER_ID are set, so it can be exercised locally against a real customer with overlapping subscriptions. The module under test is imported dynamically so the file stays import-safe while skipped. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
86f5502724 |
fix: bot-review robustness fixes (TTS sync, updater, nightly, a11y) (#4659)
Cherry-picked and re-verified the applicable subset of julianshen/readest@fa1b74a0 (its "address PR #15 bot reviews" commit). The fork-only AI-annotation-tool change was dropped — readest has no 'ai' toolbar tool. Each logic fix is covered by a failing-first test. - TTS position sequence is now an app-wide monotonic counter, so a fresh TTSController (constructed per `tts-speak`) isn't dropped by consumers holding `lastSequenceSeen` from a prior session. - share.ts only swallows AbortError (user cancel); other failures — e.g. NotAllowedError when a quick action fires without a user gesture — fall back to the clipboard so the text still reaches the user. - document.isTxt tolerates MIME params (text/plain;charset=utf-8), uppercase extensions (BOOK.TXT), and a nameless Blob, so a TXT can't slip onto the non-text path and yield a null book. - updater getNightlyPlatformKey matches x86_64/aarch64 explicitly; a 32-bit or otherwise unknown arch yields no nightly instead of mis-routing to aarch64. - UpdaterWindow downloadWithProgress resolves on tauriDownload completion even when Content-Length is absent (no more hang on portable/AppImage/Android). - nightly_update.rs uses async tokio::fs::read in the async command. - nightly.yml: serialize runs via a concurrency group (no cancel) and persist-credentials:false on checkouts. - edge TTS route only emits the word-boundary header when it fits under ~8KB; oversized values get dropped by proxies, and the client falls back to []. - RSVPOverlay drops the contradictory aria-disabled on the functional rate button (it opens the pace picker). - nightly verify harness handles artifact stream errors instead of crashing. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
1ea607829c |
fix(share): load cover under COEP, keep share links out of the clipper, fix in-app import (#4636)
Three issues found while debugging shared-book links: - /s cover was a broken <img>: the page runs under COEP: require-corp (for Turso SharedArrayBuffer), and the cover redirects to a cross-origin R2 presigned URL that can't carry a Cross-Origin-Resource-Policy header, so the browser blocked it. R2 already has CORS, but that's a different header — a plain no-cors <img> needs CORP, which presigned URLs can't set. Serve /s with COEP: credentialless, which keeps the page cross-origin isolated (the Turso replica still boots there) while allowing the image. Scoped to /s; every other route keeps require-corp. - Android share links (https://web.readest.com/s/{token}) were run through the article clipper: useClipUrlIngress excluded annotation links but not share links, so they fell through to clip_url/readability. Skip parseShareDeepLink URLs — useOpenShareLink owns that path. - In-app book import failed with "Origin null is not allowed": the importer fetched /share/{token}/download with the renderer's fetch, and on the app (tauri.localhost -> web -> R2) the second cross-origin redirect nulls the request Origin, which R2's CORS rejects. Use the native HTTP client (tauriFetch) on the app — it follows the redirect and ignores CORS, needs no server change, and works against the deployed server. Web is unaffected: its fetch's redirect to R2 is the first cross-origin hop, so the Origin is preserved and R2 allows it. Adds unit tests (middleware COEP per route, clipper skips share links, download route 302, importer uses native HTTP on app and the renderer fetch on web). Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
35b02c4efc |
feat(statistics): KOReader-compatible reading stats with cross-device sync (#4606)
Supersedes #3156. Adds a reading-statistics system whose canonical data model is KOReader's own (book + page_stat_data), so stats round-trip losslessly between Readest and KOReader. - Storage: a cross-platform Turso statistics.db in KOReader's exact schema (web/Workers, desktop, iOS, Android) — replacing #3156's Node-only better-sqlite3 + statistics.json. - Tracking: per-page reading events (time-on-page, idle-capped) flushed on page-change/idle/hide/close — the KOReader model — not session aggregates. - Sync: legacy /api/sync extended with a stats type backed by self-contained Supabase tables (stat_books, stat_pages); union/longer-duration-wins merge keyed on book_hash. apps/readest.koplugin syncs through the same endpoint. - Scale & robustness: per-tab singleton connection (avoids OPFS lock conflicts) + explicit WAL checkpoint; transactional bulk apply; chunked resumable push; client-driven paged pull with trailing-ms completion; paginated/scoped server merge. Verified: 5668 unit tests, 155 koplugin busted tests, biome+tsgo + luacheck all green; web OPFS DB verified live. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
4b0bbc77b0 |
fix(reader): open TXT files shared via "Open with" (#4571)
* fix(reader): open TXT files shared via "Open with" by converting to EPUB
The Android "Open with Readest" (VIEW intent) transient path hands the
reader the original .txt file (its filePath points at the content:// URI),
unlike the managed library which stores the already-converted EPUB. The
DocumentLoader had no branch for a raw .txt, so open() returned
{ book: null } and initViewState crashed with
"TypeError: Cannot read properties of null (reading 'metadata')",
leaving the user stuck on the library splash.
Add an isTxt() check that converts the raw .txt to EPUB in-memory (the
same TxtToEpubConverter the import path runs) and parses that. The
converter emits a .epub-named file, so the importer's own
DocumentLoader.open() on the converted file is unaffected.
Verified on-device (emulator, warm + cold start): the TXT now opens and
renders in the reader instead of crashing.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(settings): allow adjusting highlight opacity in e-ink mode
Drop the isEink prop that disabled the highlight Opacity slider under
e-ink. Opacity is still meaningful on e-ink, so let users change it.
Removes the prop from HighlightColorsEditor, its ColorPanel call site,
and the test render helper.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(send): mock fetch to fix flaky article conversion test
The article/page conversion paths fetch a favicon + author image for the
synthetic cover via globalThis.fetch. In jsdom that hit the real network:
a live fetch to the sample URL can hang up to faviconFetcher's 6s timeout,
exceeding the 5s test timeout and intermittently failing the suite. Stub
fetch so the cover falls back to its initial-letter tile (the pattern other
tests in this suite already use). Article test: ~5003ms hang -> ~80ms.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore(agent): update annotation-share-toolbar memory
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Huang Xin
|
a56cc6c61a |
feat(tts): word-by-word highlighting for Edge TTS, closes #4017 (#4566)
Highlight each word as it is spoken (Edge TTS only) instead of keeping the whole sentence highlighted, and keep the view tracking the spoken word across page boundaries. Word boundaries - Capture Edge's audio.metadata WordBoundary frames (offset/duration in 100ns ticks plus the verbatim input-text span) in the Tauri, browser, and Cloudflare-Workers WebSocket transports. - Carry boundaries through the authenticated HTTPS proxy route via an X-TTS-Word-Boundaries response header (percent-encoded JSON, ASCII-safe), so word highlighting works on the web where the browser cannot open the wss connection directly. Cache them alongside the audio blob URL. Highlighting - Sync a requestAnimationFrame loop to audio.currentTime against the boundary table and highlight the word sub-range within the spoken sentence. Synthesis stays sentence-level (natural prosody); only the visual highlight is word-level. - Suppress the sentence highlight when the active client reports word boundaries and draw the first word immediately, so the whole sentence never flashes before the first word. Fall back to the sentence highlight when a chunk has no boundaries (other engines, empty metadata). - Re-apply the current word (not the sentence) when the view relocates. Page following - Turn the page as soon as the spoken word crosses a page boundary (a tts-highlight-word event scrolls only when the word is outside the visible range), instead of waiting for the next sentence. - Check the word's position for the "back to TTS location" badge so it no longer appears while the view follows the word onto the next page. Also fixes a pre-existing bug where the browser WebSocket was constructed with an options object (valid only for the Node ws package), which threw in browsers and made the wss path unusable on the web. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Huang Xin
|
9dc41e7adf |
feat(reader): reference page numbers from EPUB page-list with manual page count fallback (#4549)
Add a 'Reference Pages' reading progress style that shows physical book page numbers in the footer progress info: - When the book carries a page list (EPUB3 nav page-list or EPUB2 NCX pageList — foliate-js already parses both and resolves the current pageItem on relocate; it was just never consumed), display the current page label and use the highest numeric label as the total, so a trailing roman-numeral index page can't corrupt the total (#672). - When the book has none, a per-book 'Reference Page Count' input appears; the reading fraction is mapped linearly onto the entered count (#4542). The count is saved per book only and never propagates to global view settings. - Falls back to percentage display when neither source is available. Verified with the sample books from #672: Caleb's Crossing (EPUB3 page-list, 419 pages) and Count Zero (EPUB2 NCX pageList/page-map, 346 pages — chapter 2 lands exactly on page 22 per its page-map), plus a stripped no-pagelist copy for the manual-count path (175/350 at 50%). Closes #672 Closes #4542 Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
loveheaven
|
11d796361e |
perf(import+open): native Rust EPUB/MOBI parser, OPF prefetch, parallel TOC enrichment (#4369)
* perf(epub): add native EPUB parser in Rust
Introduce a Rust-side EPUB pre-parser exposing three Tauri commands:
* parse_epub_metadata - title/author/cover + partialMD5 in one
shot, for the import hot path
* parse_epub_full - OPF + nav.xhtml + toc.ncx bytes plus a
manifest size table, for the reader open
hot path
* extract_epub_cover_full - full-resolution cover bytes, for the
lock-screen wallpaper writer
All three avoid ferrying multi-MB blobs across the JS<->Rust IPC
boundary. Cover bytes returned by parse_epub_metadata are downscaled
to a webview-friendly JPEG when the long edge exceeds the library
thumbnail size.
No JS callers yet -- wired up in the following commits.
* perf(import): use native EPUB parser and downscale covers on Tauri targets
On Tauri (desktop/iOS/Android), importBook now forwards EPUB
metadata + cover extraction to the Rust parse_epub_metadata
command and reuses the partialMD5 it returns, skipping the
foliate-js full archive parse and the second pass over the file
for hashing.
As a side effect, the cover written to cover.png is downscaled
to a webview-friendly JPEG (long edge <= 512px), shrinking the
on-disk thumbnail from multi-MB to ~30-60KB per book. To keep
the lock-screen wallpaper feature unchanged, useAutoSaveBookCover
now pulls the original full-resolution cover via the Rust
extract_epub_cover_full command instead of copying the (now
downscaled) cover.png; falls back to the thumbnail when the
native path is unavailable.
Web targets and non-EPUB formats keep the existing path.
* perf(reader): prefetch EPUB OPF/nav from Rust on book open
When opening an EPUB on Tauri targets, DocumentLoader now calls the
Rust parse_epub_full command up-front to pull the OPF, EPUB3 nav,
NCX and the central-directory size map in a single IPC. The
foliate-js zip loader is wrapped so that loadText() of these
entries (and a synthetic META-INF/container.xml) is served from
that in-memory cache without inflating through zip.js, while
all other assets keep flowing through the original loader.
A small in-flight dedupe is added to the spine-text loader so the
nav pipeline (loadText + createDocument back-to-back on the same
href) doesn't pay for two zip.js inflate calls per chapter on
first open.
Reader store / app service plumbing: readerStore.openBook now
resolves an absolute on-disk path via the new
appService.resolveNativeBookFilePath / bookService.resolveNativeBookFilePath
helper and threads it into DocumentLoader as nativeFilePath so
the prefetch can fire. Web targets, non-EPUB formats and books
without a managed/external on-disk path skip the prefetch and
take the original code path.
* perf(nav): parallelize section scans and memoize fragment lookups
computeBookNav now processes sections via Promise.all instead of
a sequential for-loop, and within each section issues loadText()
and createDocument() concurrently. Combined with the in-flight
loadText dedupe added to the zip loader, each chapter pays for a
single zip inflate per nav build, and the inflates of different
chapters overlap.
enrichTocFromNavElements is restructured into two concurrent
phases: a cheap '<nav' substring filter on the inflated text, and
a parsed-document walk for the survivors. Most chapters fall out
in phase 1 without ever being parsed.
In fragments.ts, calculateFragmentSize now consults a
per-section position cache (makeFragmentPositionCache) so the
N-fragment loop is O(N) over the chapter HTML instead of O(N²).
A small isCfiAddressable guard is added to skip elements that
foliate-js's CFI generator can't address (documentElement, body
itself, detached nodes, nodes outside <body>) — these previously
threw and spammed console.warn for every fragment, now they
silently fall back to the section CFI.
* perf(import): use native MOBI/AZW/AZW3 parser on Tauri targets
On Tauri (desktop/iOS/Android), importBook now forwards
MOBI/AZW/AZW3/PRC metadata + cover extraction to the Rust
parse_mobi_metadata command and reuses the partialMD5 it returns,
skipping the foliate-js full-buffer parse and the second pass over
the file for hashing. Mirrors the existing EPUB native fast-path
added in e3fc4767 — bookService tries EPUB first, then MOBI; both
bridges fall back to the foliate-js DocumentLoader when the native
path is unavailable (web target, parse error, format mismatch).
The new mobi_parser is built on the mobi crate (KF7+KF8 reader,
zero JS-side touch). It reads title, author, publisher, ISBN, ASIN,
publish date, language, subjects and description from the MobiHeader
+ EXTH records, resolves the EXTH 201 cover offset against the PDB
image-record table (with ThumbOffset / first-image fallbacks), and
strips KindleGen's HTML wrapping in EXTH 103 so the description goes
into the library DB as plain text. The parsed cover is funneled
through the same maybe_resize_cover path as EPUB, so MOBI library
thumbnails are also clamped to a 512px-long-edge JPEG.
Cover-resize / partialMD5 / RawCoverImage are extracted into a new
parser_common module shared between epub_parser and mobi_parser, so
a single tweak (e.g. raising the thumbnail target) applies to every
native importer and the partialMD5 implementation can't drift between
the two paths (a divergent algorithm would silently re-import every
existing book under a new hash on the first run).
Web targets and non-Kindle formats keep the existing path.
* test(tauri): verify native Rust EPUB parser parity with foliate-js
Add a Tauri WebView parity suite (epub-parser-parity.tauri.test.ts) that
cross-checks the native Rust parser against foliate-js on the same fixtures:
parse_epub_metadata / parse_epub_full (title, author, language, identifier,
publisher, published, subjects, partialMD5, OPF + per-entry size table), and
that opening with the native prefetch produces the same BookDoc and
computeBookNav (TOC) output as the pure-JS path.
Fix a parity divergence the suite caught: the Rust OPF parser mapped
dcterms:modified onto `published`, but foliate-js keeps them separate and
leaves `published` empty -- so EPUB3 books carrying only the mandatory
dcterms:modified got a bogus publication date on the native import path. Map
only dc:date now; add regression tests.
Test infra:
- vitest.tauri.config.mts: add optimizeDeps (mirroring vitest.browser.config)
so foliate-js-importing tauri tests load -- otherwise esbuild's dep scan
can't resolve '@pdfjs/pdf.min.mjs', pre-bundling is skipped, and the CJS
deps fail to import ("Importing a module script failed").
- capabilities-extra/webdriver.json: fix __test__ -> __tests__ fs scope typo
so import tests can open fixtures under src/__tests__/.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(import): foliate-js owns EPUB/MOBI metadata via standalone extractors
Rust contributes only the mechanical work that's expensive on a
WebView — partialMD5, the downscaled cover, and (for EPUB) the raw
OPF bytes Rust already had to read for cover resolution. Metadata
extraction is delegated to foliate-js's two new standalone entry
points (`parseEpubMetadataFromXML`, `readMobiMetadata`) so the
import-path BookDoc and the reader-path BookDoc share a single
parser implementation.
EPUB
- `parse_epub_metadata` returns
`{ partialMd5, cover, coverMime, opfPath, opfBytes }`. OPF bytes
are a free byproduct of the cover-resolution scan.
- `tryNativeParseEpub` runs `parseEpubMetadataFromXML` on the OPF
bytes and assembles a lightweight BookDoc stub (metadata +
getCover). The importer doesn't drive `DocumentLoader.open()`, so
no zip central-directory scan, no nav/ncx inflate, no spine walk.
- `coverMime` is preserved so `bookService.importBook`'s
`cover.type === 'image/svg+xml'` branch still routes SVG covers
through svg2png.
MOBI / AZW / AZW3 / PRC
- `parse_mobi_metadata` returns `{ partialMd5, cover, coverMime }`.
`tryNativeParseMobi` runs foliate's `readMobiMetadata` on the
same File, which uses `MOBI.open(file, { metadataOnly: true })`
to parse PalmDB + MobiHeader + EXTH and short-circuit before the
MOBI6 / KF8 init() that walks every text record.
- `Book.metadata.identifier` is foliate's `mobi.uid.toString()`
(PalmDB UID), the canonical MOBI identifier the reader path uses.
bookService.importBook
- EPUB and MOBI native branches consume the bridge's BookDoc stub
directly. The stub's `getCover()` returns the Rust-downscaled
blob, falling back to foliate's own `getCover` thunk when Rust
didn't extract a cover.
Other
- Drop the unused `base64` Rust dependency: cover bytes go over IPC
as `Vec<u8>` (Tauri 2 transports them natively, like opfBytes /
navBytes / ncxBytes).
- Drop the `nativePrefetch` option on `DocumentLoaderOptions`; no
caller passes it. `nativeFilePath` keeps driving `parse_epub_full`
on the open hot path.
Tests
- vitest.tauri parity test asserts byte-equal partialMD5, cover
presence parity, OPF bytes that decode to a real `<package>`
document, and that `parseEpubMetadataFromXML` on those bytes
produces the same user-visible metadata fields (title / author /
language / identifier / published) as `DocumentLoader.open()`.
* test(tauri): add War and Peace MOBI fixture for native parser parity
The .tauri parser-parity suite previously had no .mobi/.azw3 asset, so the native MOBI parser (metadata + EXTH cover resolution) was uncovered. Adds a real KF8 MOBI ("War and Peace") to enable MOBI parity coverage against foliate-js.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(foliate-js): bump submodule to readest/foliate-js main (91191ca)
Replaces the ad-hoc 02f435a with the merged main commit 91191ca, which lands the standalone OPF/MOBI metadata extractors (parseEpubMetadataFromXML, readMobiMetadata) the import fast-path depends on (foliate#19), plus the RTL multi-view rect-mapper fix (foliate#20). The extractor code is byte-identical to 02f435a, so the bridges are unaffected.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Huang Xin <chrox.huang@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
Huang Xin
|
e29331bea9 |
fix(sync): prevent cross-device progress overwrite; retry first pull on flaky networks (#4341)
Closes #4222. Three changes that together stabilize Readest sync across devices: **Stop the artificial updatedAt bump in useProgressAutoSave** saveConfig unconditionally bumps config.updatedAt = Date.now(), and useProgressAutoSave used to fire saveConfig on the very first relocate after book open — even though that relocate just reflects the position loaded from disk, not user action. The stale local config then looked "newer" than a fresher server-side push, so the next auto-push overwrote the other device's real progress via last-writer-wins. The hook now snapshots the loaded location and skips saveConfig when the in-memory location still matches it. **Retry the first config pull with backoff + release the gate** useProgressSync gates pushes behind a successful pull (so a brand-new import can't clobber the server's real progress). But handleAutoSync only re-arms on progress.location changes, so a single failed pull (Android cold-start contention, Wi-Fi/LTE handoff, captive portal) used to block every push for the whole reader session. The new pullWithRetry retries on backoff (1500/4000/10000 ms) and releases the gate after exhaustion — server-side last-writer-wins still protects the cross-device case (a stale local push with an older updated_at loses to a fresher server record). sync-book-progress events reset the chain so manual pull-to-refresh recovers cleanly. Fetch timeout bumped from 8s to 15s to better tolerate slow networks in that cold-start window. **Server piggybacks books.progress off configs push** /api/sync POST now updates books.progress + books.updated_at for each upserted config, gated by .lt('updated_at') so a concurrent newer books push is never downgraded and a missing row is a silent no-op (useBooksSync still seeds new rows from the library page). The in-reader syncBooks round-trip is dropped — the reader now sends one POST per auto-save instead of two, and the books row stays consistent with config pushes even while a reader stays open (#4198). Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
loveheaven
|
b5d898a48e |
fix(document): accept zips whose magic bytes are mangled but have valid EOCD (#4321)
Some EPUBs (e.g. downloaded from Baidu Netdisk) have their first few bytes corrupted while the rest of the archive is still well-formed. The zip spec locates archives via the End-of-Central-Directory record at the file tail, so a corrupted local file header signature does not actually invalidate the file. When the leading magic check fails, fall back to scanning the last ~64 KiB for the EOCD signature (PK\x05\x06); if present, treat the file as a zip and let zip.js read it normally. |
||
|
Huang Xin
|
1eae2af23e |
feat(sync): batch replica sync into one /api/sync/replicas request (#4109)
Auto-sync triggers (boot non-settings, focus, visibilitychange, online,
periodic) used to fan out N parallel `GET /api/sync/replicas?kind=…`
requests, one per replica kind. With 5 kinds today and the focus path
firing on every foreground transition, that's 5x the Cloudflare Worker
invocations of what the work actually requires.
Server: extend POST /api/sync/replicas to accept a batched-pull body
(`{ cursors: [{kind, since}, …] }`) alongside the existing push
(`{ rows: […] }`). Per-kind queries fan out via Promise.all — Supabase
calls inside the Worker aren't billed as Cloudflare requests, so DB
load is unchanged while Worker invocations collapse from N to 1.
Client/manager: add `client.pullBatch` and `manager.pullMany` that
share the existing cursor/HLC machinery. The boot path's `since=null`
override carries over via `pullMany(kinds, { since: null })`.
Orchestrator: `triggerIncrementalPullAll` now does ONE pullMany call
then fans out per-kind apply via Promise.allSettled. Boot does the
same for non-settings kinds (settings stays a single call to preserve
its apply-first ordering invariant).
Foreground triggers: listen to BOTH `focus` and `visibilitychange`,
sharing one throttle. focus is fastest on iOS Tauri WKWebView (~T=0,
~400ms ahead of visibilitychange). visibilitychange is the only
signal that fires on browser tab switching — focus does not. Drops
the Supabase user-ref-change listener (was the slowest of the three
foreground signals; redundant with the DOM events).
Bonus: `useBooksSync` now serializes `handleAutoSync` against
`pullLibrary` via the shared `isPullingRef` gate. The two paths used
to fire two concurrent `/api/sync?type=books` requests on the same
`since` value at startup; now whichever runs first claims the gate
and the other skips (throttle's `emitLast` retries afterwards).
Per session: boot 5→2 Worker calls. Per foreground trigger: 5→1.
|
||
|
Huang Xin
|
c30a59a9ed |
fix(epub): accept EPUBs with malformed first ZIP local file header (#4103)
Some EPUB writers (e.g., "ebookredo") emit a non-standard local file header signature on the first entry — bytes like PK\x03\x02 instead of PK\x03\x04. The archive is still readable via the End of Central Directory record, and @zip.js/zip.js handles it without complaint, but DocumentLoader.isZip() rejected the file at the magic-bytes gate before zip.js ever ran. The user saw "Unsupported or corrupted book file" on a perfectly readable EPUB. Drop the strict 4th-byte equality check. PK\x03 alone identifies a local file header — no other ZIP record signature starts that way — so loosening the check is safe and aligns with what tolerant ZIP parsers already do. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
6e7c9d1395 |
feat(sync): bundled settings replica kind for cross-device prefs and credentials (#4094)
* feat(sync): add bundled `settings` replica kind for cross-device prefs and credentials Adds a single-row `settings` replica that syncs a whitelist of `SystemSettings` fields across devices via per-field LWW (one entry per dot-namespaced path). Plaintext for theme / highlight colour / TTS configuration; encrypted (AES-GCM under the user's sync passphrase) for kosync / Readwise / Hardcover credentials. Highlights: - Push-side diff against an in-memory snapshot for plaintext paths and a localStorage SHA-256 hash for encrypted paths, so a refresh doesn't re-publish or re-prompt for the passphrase. - Pull-side cipher-fingerprint dedupe + per-row passphrase gate; decryption failures surface as toasts (wrong passphrase / orphan cipher) instead of silent drops. - Auto-recovery for orphaned ciphers: when a row references a saltId no longer in `replica_keys`, clear the local hash and re-encrypt under the current salt on the next save. - Single in-flight `/sync/replica-keys` fetch with a value cache to coalesce the boot-time burst of concurrent unlock callers. * fix(sync): guard settings dot-path helpers against prototype-polluting keys Reject `__proto__`, `constructor`, and `prototype` segments in the settings adapter's `readPath` / `writePath`. Every caller currently passes a constant from `SETTINGS_WHITELIST`, so the guard is purely defensive — but it silences the CodeQL prototype-pollution warning on PR #4094 and keeps the helpers safe if a future call site ever forwards an untrusted path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
2d5590ec1f |
feat(applock): 4-digit PIN gate at app launch (#4093)
Closes #2285. Adds an opt-in 4-digit PIN that gates the library and reader on app launch. Threat model: casual physical/browser access by another person on a shared device — peace of mind, not defense against an attacker with filesystem access. The PIN is stored as a salted PBKDF2-SHA256 hash (100k iterations) in settings.json; the plaintext PIN is never persisted. Configured from Settings → Advanced Settings → "Set PIN…" (and "Change PIN…" / "Disable PIN…" once enabled). The lock screen and the set/change/disable dialog share a single 4-dot input component (PinInput) for a consistent UI; the dialog auto-advances focus from Current → New → Confirm. Lock-on-resume, biometric unlock, and account-based reset are out of scope for this MVP — disable for now is "clear app data". Bundles the previously-missed sync-passphrase i18n strings (PR #4090) across all 33 locales so no `__STRING_NOT_TRANSLATED__` placeholders remain in the tree. New - src/libs/crypto/applock.ts (PBKDF2 hash/verify; reuses derivePbkdf2Key) - src/store/appLockStore.ts (gate + dialog state) - src/components/PinInput.tsx (shared 4-dot input) - src/components/AppLockScreen.tsx (full-screen lock gate) - src/components/settings/AppLockDialog.tsx (set/change/disable) - src/__tests__/libs/crypto/applock.test.ts Modified - src/types/settings.ts (pinCodeEnabled / pinCodeHash / pinCodeSalt) - src/services/constants.ts (default off) - src/components/Providers.tsx (mount gate + dialog above app shell) - src/app/library/components/SettingsMenu.tsx (Advanced submenu entries) - src/styles/globals.css (animate-pin-shake keyframe) - public/locales/*/translation.json (21 PIN keys + 17 leftover passphrase keys × 33 locales) Verified - pnpm test (4018 pass) - pnpm lint (clean) - Manual web smoke: Set/Reload-locks/Wrong-PIN/Unlock/Change/Disable Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
712d564e9d |
feat(sync): encrypted OPDS credentials + Tauri keychain (PR 4c + 4d) (#4090)
* feat(sync): encrypt opds_catalog credentials end-to-end (TS path) Wires encrypted-credential sync for opds_catalog via the CryptoSession shipped in PR 4a (#4084) plus a new publish/pull crypto middleware. TS-only — native still uses ephemeral storage (re-enter passphrase per launch); PR 4d wires the OS keychain. - ReplicaAdapter gains optional `encryptedFields: readonly string[]`. Adapters stay sync; the middleware handles the crypto round trip. - replicaCryptoMiddleware.ts: encryptPackedFields drops the named fields from the push when the session is locked (no plaintext leak); decryptRowFields drops them on pull failure (local plaintext preserved by the store merge). - replicaPublish / replicaPullAndApply invoke the middleware. - OPDS adapter declares encryptedFields = [username, password] and now pack/unpack them as plaintext. - passphraseGate.ts: ensurePassphraseUnlocked coalesces concurrent calls, prompts via the registered prompter with kind=setup|unlock, throws NO_PASSPHRASE on cancel. - PassphrasePromptModal mounted at the Providers root; registers itself as the gate prompter. - CryptoSession.forget() wipes server-side envelopes + salts. - Migration 010 + replica_keys_forget RPC; DELETE /api/sync/replica-keys + client wrapper. - SyncPassphraseSection on the user page: status / Set / Unlock / Lock / Forgot. - CatalogManager pre-save: ensurePassphraseUnlocked when credentials are present; user cancel saves locally without sync. Plan updated: PR 4 split documented as 4a/4b/4c/4d. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(sync): persist sync passphrase via OS keychain (Tauri) Replaces the EphemeralPassphraseStore stub on native with real OS-keychain storage so users don't re-enter their sync passphrase every launch. Web stays on the in-memory ephemeral store by design. Native bridge plugin gains 4 commands wired across all platforms: - Rust desktop (`keyring` crate): macOS Keychain on apple-native, Windows Credential Manager on windows-native, Linux libsecret/ Secret Service on sync-secret-service. Per-target features so each platform compiles only the backend it needs. - iOS Swift: Security framework Keychain (kSecClassGenericPassword, SecItemAdd / Copy / Delete). - Android Kotlin: androidx.security EncryptedSharedPreferences (AndroidKeystore-derived AES-GCM master key, AES256_SIV / AES256_GCM key/value encryption). TS layer: - TauriPassphraseStore wraps the bridge calls. set is fail-loud (surfaces keychain rejection); get is fail-soft (returns null on any error so the gate prompts). - createPassphraseStore returns ephemeral synchronously; upgradeToKeychainIfAvailable swaps the singleton to TauriPassphraseStore on Tauri after probing the bridge. CryptoSession resolves the store via createPassphraseStore() each touch so the swap is transparent. - CryptoSession.tryRestoreFromStore: silent unlock at boot. Stale- entry recovery clears the store when the account has no salt server-side. unlock/setup persist; forget also clears the store. - Providers boot effect: upgrade keychain → silent restore. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(sync): make encrypted-credential pull actually decrypt + UX polish PR 4c shipped the encrypt path but the pull side silently dropped ciphers when locked, the modal was busy with double-rings, and web re-prompted on every page refresh. This rolls up the post-test fixes + UX polish: Pull-side decrypt: - decryptRowFields takes an `onLocked` callback the orchestrator wires to the passphrase gate; encountering a cipher field with a locked session now triggers the lazy-prompt path instead of dropping the field. - replicaPullAndApply re-applies the unpacked row for metadata-only kinds even when a local copy exists, so the now-decrypted creds reach the store (the binary-kind skip-if-local optimization doesn't apply). - Cipher fingerprint comparison: capture the row's `cipher.c` for each encrypted field, compare against the local record's lastSeenCipher. Same → skip prompt + decrypt entirely. Different (rotation / value change on another device) → prompt to re-decrypt. Fingerprint persists via OPDSCatalog.lastSeenCipher. Web persistence: - SessionStoragePassphraseStore: passphrase survives page refresh within the same tab, dies on tab close. Replaces EphemeralPassphraseStore as the default on web. Avoids localStorage / IndexedDB to keep the tab-scoped trust boundary. UI: - Renamed PassphrasePromptModal → PassphrasePrompt; modernized: filled input style with single subtle focus border, btn-primary + btn-ghost replaced with leaner custom buttons. eink-bordered + btn-primary classes give the dialog correct e-paper rendering. - globals.css: suppress redundant outline/box-shadow on focused text inputs / textareas (the element's own border is the focus indicator). - AGENTS.md: documents the e-ink convention (`eink-bordered`, `btn-primary` for inverted CTAs, etc.) so future widgets ship with e-paper support. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
6bfeb295d2 |
feat(sync): add opds_catalog replica kind (plaintext fields) (#4087)
Wires OPDS catalogs through replica sync as a metadata-only kind.
Plaintext fields only in this PR — encrypted credentials (username,
password) ship in the follow-up alongside the SyncPassphrasePanel UI
and Tauri keychain backend.
- Migration 009 extends the kind allowlist with 'opds_catalog'.
- replicaSchemas adds opdsCatalogFieldsSchema (name, url, description,
icon, customHeaders, autoDownload, disabled, addedAt) with a 50-row
per-user cap.
- New opdsCatalogAdapter is metadata-only (no `binary` capability).
Stable cross-device id from md5("opds:" + url.lower()) so two
devices that import the same URL converge to one row instead of
duplicating.
- New customOPDSStore (zustand) hydrates from SystemSettings,
publishes upserts/deletes through the replica pipeline, preserves
local-only username/password when overlaying remote updates, and
strips tombstones at the persistence boundary so existing
useSettingsStore readers (useOPDSSubscriptions, pseStream,
app/opds/page.tsx) need no migration.
- replicaPullAndApply branches on adapter.binary so metadata-only
kinds skip the bundleDir requirement and the manifest/binary path.
- CatalogManager rewires Add / Edit / Remove / Toggle / Add-popular
through the new store.
Plan update bundled in: tenet 8 (scalar settings sync via a bundled
row; collections sync per-record), per-kind allowlist now includes a
`settings` singleton that will collapse PRs 5 + 6+ into one bundled
adapter, and PR 4 is split into 4a (already merged) / 4b (this) / 4c
(encrypted credentials + UX).
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Huang Xin
|
35227ecd61 |
feat(sync): wire crypto session for encrypted-field sync (#4084)
Adds the per-account PBKDF2 salt endpoint and an in-memory CryptoSession
that derives keys lazily per saltId. Lets future kinds (OPDS catalogs in
PR 4b) encrypt/decrypt fields without re-deriving on every operation.
- Migration 008: replica_keys_{create,list} RPCs round-trip the bytea
salt as base64; SECURITY INVOKER, RLS-gated by the existing replica_keys
policies.
- /api/sync/replica-keys GET/POST endpoint matches the dual app/pages
shape used by /api/sync/replicas.
- ReplicaSyncClient.{listReplicaKeys,createReplicaKey} wraps the endpoint.
- CryptoSession.{unlock,setup,encryptField,decryptField,lock} caches
derived keys per saltId; foreign envelopes trigger a lazy re-list +
derive. Iterations injectable so tests run with PBKDF2 ITER=1000.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Huang Xin
|
de6529523f |
feat(sync): cross-device background texture sync (#4079)
Plug the texture replica adapter into the kind-agnostic primitives shipped in #4077. Textures imported on one device download and become available on every signed-in device, with the same shape as the font sync stack (single-file binary, contentId from partialMD5+size+filename, bundleDir layout, replica-publish on import, full activation on auto-download). Includes legacy flat-path migration so pre-existing textures sync without re-import, ColorPanel import flow now publishes the row and queues the binary upload, and createCustomTexture preserves contentId/bundleDir/byteSize through addTexture (mirrors the font-import fix). Server allowlist gains 'texture' with a single-image Zod schema; useBackgroundTexture passes replica metadata through addTexture so the boot-time "ensure selected texture is in store" path doesn't silently un-publish a remote record. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
981579c255 |
feat(sync): cross-device custom font sync (#4077)
* refactor(sync): kind-agnostic replica primitives Extract dict-only sync into shared primitives (registry, pull/apply orchestrator, persist env, schema allowlist) so other kinds can plug in. Companion changes: per-replica Storage Manager grouping, useReplicaPull boot-race recovery, manifest=null reconciliation on every boot pull, copyFile takes explicit srcBase + dstBase, settled- event helpers, lenient webDownload Content-Length (R2/S3 signed URLs commonly omit it), and generic "File" transfer toast copy any replica kind can share. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(sync): cross-device custom font sync Plug the font replica adapter into the kind-agnostic primitives: font store gains replica wiring, custom font import publishes the replica row + queues a binary upload, and bootstrap registers the font adapter and download-complete handler. Includes legacy flat-path migration so pre-existing fonts sync without re-import, full @font-face activation on auto-download (load + mount the rule, mirroring manual import), and a fix to createCustomFont so contentId / bundleDir / byteSize survive the trip through addFont — otherwise import-time publish silently no-oped on missing contentId. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
cbdc3b8f52 |
feat(sync): wire dictionary store through replica sync (follow-up to #4075) (#4076)
* feat(sync): cross-device dictionary sync Custom MDict / StarDict / DICT / SLOB dictionaries now sync across signed-in devices via the replica layer. - Store mutations publish replica rows with field-level LWW + tombstones. - Re-importing the same content (renamed or after delete) preserves the user's label and reincarnates the server row instead of duplicating. - Manifest commits after binary upload so other devices never see a row whose binaries aren't on cloud storage yet. - Pull-side orchestrator creates a placeholder dict, queues the binaries via TransferManager, and clears the unavailable flag on completion. - Toast copy branches by transfer kind so dict uploads don't read "Book uploaded". Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(sync): boot pull and binary download path - Defer the boot pull until TransferManager is initialized so download enqueues aren't dropped. - Auto-persist the local dict store after applyRemoteDictionary; otherwise the next loadCustomDictionaries wipes the in-memory rows. - Boot pull passes since=null so a device whose cursor advanced past unpersisted rows can still recover. - Skip pulling when not authenticated instead of logging "SyncError: Not authenticated" on every boot of a signed-out device. - downloadReplicaFile resolves the destination against the kind's base dir; binaries previously landed at the literal lfp and openFile then failed with "File not found". Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(sync): per-page useReplicaPull hook Lifts the boot-time pull out of EnvContext into a hook each page mounts for the kinds it needs: useReplicaPull({ kinds: ['dictionary'] }). Library page and the shared Reader component opt in. The hook fires 10s after page load (so feature mounts hydrate first), dedups per-kind across navigation, and releases the slot on failure so a later mount can retry. Future kinds plug into the hook's per-kind switch. Also closes two refresh-loop bugs: - Hydrate the dict store from settings BEFORE the apply loop, so the auto-persist doesn't clobber persisted rows that the in-memory store hadn't yet read. Library-page refresh was the visible victim. - Skip the download queue when every manifest file is already on disk under the resolved bundle dir. Refreshing is a no-op; partial- download recovery still queues because some files would be missing. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
3b348c8f35 |
feat(sync): CRDT replica sync foundation (#4075)
* feat(sync): foundation for CRDT-based cross-device replica sync (Phase 1+2)
Adds the primitives and orchestration layer for syncing user-imported
assets (dictionaries, fonts, textures, OPDS catalogs, dict settings)
across devices via a polymorphic `replicas` table with field-level LWW
under HLC ordering. Phase 1 ships the foundation (CRDT, crypto, server
schemas, SQL migrations, push/pull endpoint); Phase 2 adds the adapter
registry, HTTP client, and sync manager. No modifications to existing
book sync — additive only.
Phase 1:
- src/libs/crdt.ts — HlcGenerator (monotonic + remote-absorption +
clock-regression-safe), per-field LWW with deviceId tiebreak,
remove-wins tombstones, reincarnation token revival.
- src/libs/crypto/{derive,encrypt,envelope,passphrase}.ts —
PBKDF2-600k key derivation (OWASP 2024), AES-GCM round-trip,
envelope {c,i,s,alg,h} with SHA-256 sidecar integrity check,
passphrase storage abstraction (web ephemeral; Tauri keychain stub).
- src/libs/replica-schemas.ts — Zod-backed allowlist (dictionary only
in PR 1), 64KiB row cap, 64-field cap, schemaVersion bounds,
filename validator.
- src/libs/replica-sync-server.ts — push batch validation
(auth + allowlist + schema + HLC ±60s skew clamp).
- src/pages/api/sync/replicas.ts — POST/GET endpoint wrapping the
Postgres crdt_merge_replica function via RPC.
- docker/volumes/db/migrations/003_add_replicas.sql — replicas table
+ replica_keys table + RLS.
- docker/volumes/db/migrations/004_crdt_merge_replica_fn.sql — atomic
per-field LWW merge function (forwards-compat preserves unknown
fields).
Phase 2:
- src/services/sync/replicaRegistry.ts — adapter contract
(core + optional BinaryCapability + LifecycleHooks per eng review).
- src/libs/replica-sync-client.ts — HTTP wrapper mapping status codes
to typed SyncError codes.
- src/services/sync/replicaSyncManager.ts — 5s debounced push,
immediate flush on visibilitychange/online, per-kind pull cursor,
remote HLC absorption.
Tests: 125 new (crdt 26, crypto 32, schemas 21, server 16, client 12,
registry 6, manager 12). Full suite 3656 passing, lint clean. Existing
book/config/note sync paths untouched.
Plan: ~/.claude/plans/vivid-orbiting-thimble.md
CEO plan: ~/.gstack/projects/readest-readest/ceo-plans/2026-05-06-replica-sync-cathedral.md
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(sync): add kind="replica" path through TransferManager (Phase 3)
Adds the replica branch to the existing book-shaped transfer
infrastructure so dictionary (and future kinds) bundles can flow through
the same queue, retry, and progress UI as book uploads.
Existing book transfer paths remain unchanged. The book-side regression
suite (37 tests in transfer-store.test.ts, 37 in transfer-manager.test.ts)
all stay green.
Store (src/store/transferStore.ts):
- TransferItem gains kind: 'book' | 'replica' (default 'book' on legacy
persisted rows), replicaKind, replicaId, replicaFiles, replicaBase.
- New addReplicaTransfer(replicaKind, replicaId, displayTitle, type, opts)
with files + base in opts; auto-computes totalBytes from file sizes.
- New getReplicaTransfer(replicaKind, replicaId, type) lookup.
- getTransferByBookHash filters to kind === 'book' (defensive against
bookHash="" collisions on replica items).
- restoreTransfers fills kind: 'book' for legacy persisted rows.
Manager (src/services/transferManager.ts):
- queueReplicaUpload / queueReplicaDownload / queueReplicaDelete.
- executeTransfer dispatches by kind to the new executeReplicaTransfer
(iterates files, calls appService.uploadReplicaFile per file with
per-file progress aggregation) or the existing executeBookTransfer
(refactored out, byte-identical behavior).
- Dispatches replica-transfer-complete event on success so stores can
react (e.g., commit manifest_jsonb to the replica row).
Storage / cloud (src/libs/storage.ts, src/services/cloudService.ts):
- uploadReplicaFile bypasses the book-only File.name smuggling and
takes an explicit cfp (cloud file path).
- uploadReplicaFileToCloud / downloadReplicaFileFromCloud /
deleteReplicaBundleFromCloud orchestrate per-file operations under
${userId}/Readest/replicas/<kind>/<replicaId>/<filename>.
- replicaCloudKey() centralizes the path-construction rule.
- New CLOUD_REPLICAS_SUBDIR constant.
App service (src/services/appService.ts, src/types/system.ts):
- AppService gains uploadReplicaFile, downloadReplicaFile,
deleteReplicaBundle (file-level operations; orchestration lives in
TransferManager).
Tests: 18 new (12 in transfer-store.test.ts, 5 in transfer-manager.test.ts,
1 fixture). Full suite 3674 passing, lint clean. Existing book regression
clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(sync): dictionary replica adapter + bootstrap (Phase 4, partial)
Lands the safe-to-ship foundation of Phase 4 — adapter logic, registry
bootstrap, and SystemSettings hooks for replica sync. The on-disk
migration of legacy customDictionaries (bundleDir → content-hash id),
the live store wiring, and the Settings → Sync UI are deferred to a
follow-up PR so they can land with real-device QA.
Adapter (src/services/sync/adapters/dictionary.ts):
- dictionaryAdapter: kind='dictionary', schemaVersion=1.
- pack/unpack — only synced subset (name, kind, lang, addedAt,
unsupported{,Reason}). bundleDir / files / unavailable / deletedAt
stay per-device or are handled by the tombstone mechanism.
- BinaryCapability.enumerateFiles dispatches by bundle kind:
- mdict: mdx + mdd[] + css[]
- stardict: ifo + idx + dict + syn (skips .idx.offsets / .syn.offsets
sidecars — those are device-local indices)
- dict: dict + index
- slob: single .slob file
- primaryDictionaryFile() picks the anchor file per kind for
partialMD5 hashing.
- computeDictionaryReplicaId(partialMd5, byteSize, sortedFilenames)
produces a deterministic 32-hex content-hash id used at import time.
- 23 tests cover pack/unpack identity, kind dispatch, file enumeration,
id determinism, and per-device-field exclusion.
Bootstrap (src/services/sync/replicaBootstrap.ts):
- bootstrapReplicaAdapters() registers all known adapters once at app
start. Idempotent (safe to call multiple times). Wired into
EnvContext.tsx so the registry populates on app mount.
- 3 tests cover registration, idempotency, and the PR-1 allowlist.
SystemSettings (src/types/settings.ts, src/services/constants.ts):
- +SyncCategory = 'book' | 'progress' | 'note' | 'dictionary' — typed
union for the user-facing sync toggles. 'progress' gates the
existing book-config sync (reading progress); 'note' gates
annotations; 'book' gates book binaries + metadata; 'dictionary'
gates the new replica sync. Future replica kinds extend the union.
- +SYNC_CATEGORIES readonly array for UI iteration.
- +syncCategories: Partial<Record<SyncCategory, boolean>> — per-
category opt-in toggles in DEFAULT_SYSTEM_SETTINGS (default ON for
all four). UI panel ships in the follow-up.
- +lastSyncedAtReplicas: Record<string, string> — per-kind HLC pull
cursors (matches replicaSyncManager's CursorStore contract).
Registry type cleanup (src/services/sync/replicaRegistry.ts):
- BinaryCapability.enumerateFiles return shape: localRelPath → lfp
to match the existing TransferStore.ReplicaTransferFile convention.
Tests: 28 new (23 dict + 3 bootstrap + 2 syncCategories defaults).
Full suite 3702 passing, lint clean. Existing book/config/note sync
paths untouched.
Deferred to PR 1 follow-up (with real-device QA):
- customDictionaryStore migration: rehash legacy uniqueId() bundleDir
to content-hash id; preserve providerOrder mapping; staged
.legacy/<old-id>/ backup.
- Wire customDictionaryStore mutations through replicaSyncManager
(markDirty on add/rename/delete; pull on init).
- Settings → Sync panel: per-category toggles + last-sync timestamps.
- Sync passphrase modal: set / change / forgot flow (lazy first prompt
on encrypted-field push/pull).
- <CloudReplicaRow> in CustomDictionaries.tsx for "Download from cloud
(X MB)" affordance.
- Tauri keychain backend for sync passphrase storage.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(sync): replicaSync singleton + content-hash id at dict import (Phase 4b)
Two foundation pieces that everything UI-side will sit on top of, both
purely additive — no live store wiring, no behavior change for existing
dictionary imports.
replicaSync singleton (src/services/sync/replicaSync.ts):
- initReplicaSync({deviceId, cursorStore, hlcStore?, client?}) builds
one ReplicaSyncManager backed by an HlcGenerator with persistence
wrapped around .next()/.observe(). Idempotent (second init returns
the existing instance).
- LocalStorageHlcStore (src/libs/hlc-store.ts) snapshots the HLC
counter under 'readest_replica_hlc' so it survives restart. Falls
back silently when localStorage is unavailable (private mode, SSR);
client re-derives via the existing remote max(updated_at_ts) repair
path. InMemoryHlcStore is the test backend.
- 16 tests (9 hlc-store, 7 replicaSync) covering snapshot persistence,
restore-on-init, and idempotency.
- Wiring into EnvContext for production deferred to the follow-up that
also adds the cursor store backed by useSettingsStore.
Content-hash id at dictionary import
(src/services/dictionaries/contentId.ts):
- computeDictionaryContentId(primaryFile, filenames) wraps
computeDictionaryReplicaId(partialMd5(primary), byteSize,
sortedFilenames) — the cross-device id used as the replica_id when
the dict actually pushes/pulls.
- Wired into all four import paths in dictionaryService.ts:
- stardict primary = .ifo (small text, partialMD5 ≈ full hash)
- mdict primary = .mdx (body)
- dict primary = .dict.dz (gzipped body)
- slob primary = .slob (single-file bundle)
ImportedDictionary gains contentId?: string. Optional for backwards
compat; legacy bundles without contentId are flagged as
"needs rehash before sync" by the upcoming store-wiring follow-up.
- 6 tests cover identity determinism, byteSize sensitivity, filename-
set sensitivity, and order-independence.
Tests: 22 new (9 + 7 + 6). Full suite 3724 passing, lint clean.
Existing dictionary import flow unchanged for users — contentId is an
additional field, not a replacement for the bundleDir-based id.
Deferred to follow-up (with on-device QA):
- Production cursor store backed by useSettingsStore +
appService.saveSettings.
- EnvContext call to initReplicaSync after appService boot.
- customDictionaryStore mutation hooks → replicaSyncManager.markDirty.
- Legacy bundleDir → contentId migration with .legacy/ backup.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(sync): land replica-sync design plan in repo
Moves the plan document that drove this PR's foundation work
(`~/.claude/plans/vivid-orbiting-thimble.md`) into the project tree at
`apps/readest-app/.claude/plans/` so reviewers and future contributors
can read it alongside the code without leaving the repo.
The plan went through three review passes — Codex (19 findings, all
absorbed), CEO/scope review (mode SCOPE EXPANSION; encrypted secrets
pulled forward to v1, "private-only forever" posture lock), and eng
review (FULL_REVIEW mode, 16 findings absorbed). The full review trail
lives in the file's `## GSTACK REVIEW REPORT` section at the bottom.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(crdt): point README at in-repo plan path
Now that vivid-orbiting-thimble.md lives at
apps/readest-app/.claude/plans/, the README link should point there
rather than at the home-dir copy that no longer exists.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(sync): rename src/libs files to camelCase per project convention
Test files renamed in lockstep. Imports + comment references updated
across cloudService, storage, transferManager, replicaSync,
replicaSyncManager, /api/sync/replicas, and all four test files. No
behavior change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Huang Xin
|
f5657fb3a0 |
fix(share): correct recipient import flow and assorted UI polish (#4043)
- ensureSharedBookLocal helper makes sure the local library has both the Book entry and the bytes on disk after /import succeeds; navigating into the reader before this lands on "Book not found" - ShareLanding navigates via navigateToReader (path form on web) so the reader actually renders instead of hitting the App Router stub and going blank - Loading + progress UI on the landing page while bytes stream in; Open-in-app disabled mid-import to avoid races - UserInfo header: vertically center avatar with name/email, tighter mobile gap, and a fillContainer prop on UserAvatar so a parent can size the box via classes without the inline style fighting back - Rename "Share current page" -> "Share reading progress" (and matching post-generation hint) and shrink the dialog from 480 to 460px - Drop the unused Reload Page menu item from SettingsMenu - Translate the two new i18n keys across 31 locales Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
d1e7b4902c |
feat(share): time-limited share links with cfi-aware imports (#4037)
Add a Share Book feature that generates an expiring HTTPS share URL plus a
parallel readest://share/{token} deep link. Recipients land on /s/{token},
where logged-in users can one-tap "Add to my library" (R2 server-side
byte-copy) and anonymous users download the book or open it in the app.
Sharers manage active links from a dedicated "Manage Shared Links" panel
under user settings.
Highlights:
- 9 new App Router endpoints under /api/share (create, [token], cover,
og.png, download, download/confirm, import, revoke, list).
- /s landing page with branded next/og chat unfurl image and SSR auth-cookie
detection so logged-in recipients see "Add to my library" as the primary
action without layout shift.
- Server-side R2 byte-copy for /import preserves the project's existing
invariant that every files.file_key is prefixed with its row's user_id;
stats / purge / delete / download routes work unchanged. URL-encodes the
copy source so titles with spaces or '&' don't break the copy.
- Universal 7-day expiry cap, no tier differentiation, no "never". DMCA-risk
reduction. Picker defaults to 3 days.
- Position-aware shares: "Share current page" toggle (off by default for
privacy) attaches the sharer's CFI; recipient lands at the same paragraph.
- Per-user 50-share cap, rate limiting via Cache-Control: no-store on
token-bearing responses, atomic SQL increment for download_count via a
SECURITY DEFINER function so the public confirm beacon stays safe under
concurrent fire.
- Soft revocation: presigned download URLs (5-min TTL) cannot be cancelled
before TTL; documented as accepted v1 behavior.
- token + token_hash hybrid storage: public endpoints look up by hash and
never select the raw token, so accidental SELECT-* leakage on a public
route can't expose the bearer credential.
- Mobile / desktop Tauri share via tauri-plugin-sharekit; web falls back to
navigator.share with a clipboard fallback when no native share method
exists. Share-sheet dismissal no longer silently copies.
UI:
- New Dialog with a settings-card group: iOS-style segmented duration picker
+ toggle slider for "Share current page", on a single row each.
- Reader top-bar Share button, library context-menu Share entry, manage-
shares list with cover thumbnails and overflow menu.
- New <SegmentedControl> primitive in src/components for reuse.
Coverage:
- Unit tests for token utils + URL parser (20 new tests, full suite at 3445).
- 31 locales translated for all new strings; en plurals hand-added per the
project's hand-curated en convention.
DB migration in docker/volumes/db/migrations/002_add_book_shares.sql adds
the book_shares table, RLS policies, and the increment_book_share_download
RPC. Migration is idempotent.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Huang Xin
|
234ecc3113 |
fix(epub): fall back to case-insensitive zip lookups (#3991)
Try an exact ZIP entry match first, then fall back to a case-insensitive lookup for EPUB resources when archive entry casing differs from manifest paths. Keep ambiguous case-only duplicates exact-match only, and add a regression test that opens a real EPUB through DocumentLoader. |
||
|
fabrisnick
|
b251e537d3 |
chore(deps): update stripe (#3941)
* chore(deps): update stripe * fix: filter and sort not affecting purchases Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * refactor: rename restoredPurchases to restoredSubscriptions The filter excludes one-time purchases, so the array contains only subscriptions. Naming reflects that. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Co-authored-by: Huang Xin <chrox.huang@gmail.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
b0cc5461af |
refactor(toc): cache navigable structure per book (#3869)
* refactor(toc): cache TOC + section fragments per book
Moves the TOC regrouping and section-fragment computation out of
foliate-js/epub.js #updateSubItems into the readest client as
computeBookNav / hydrateBookNav in utils/toc.ts. The result is
persisted to Books/{hash}/nav.json — capturing the book's full
navigable structure (TOC hierarchy + sections with hierarchical
fragments). Compute once, persist locally, hydrate on subsequent
opens. Designed to serve current human-facing navigation (TOC
sidebar, progress math) and future agentic navigation (LLM-driven
seeking by structural location).
Versioned by BOOK_NAV_VERSION for forward invalidation. Existing
books regenerate transparently on next open.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: update worktree scripts
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
Huang Xin
|
3df75a67f9 | feat(tts): support edge tts on cloudflare worker (#3819) | ||
|
Huang Xin
|
caa0d719c5 | compat(vertical): check writing mode also for child element of body, closes #3583 (#3743) | ||
|
AK Venugopal
|
80cab8e56d |
feat: Hardcover.app Sync (#3724)
* feat(hardcover): add one-way Hardcover sync integration - Add HardcoverClient with ISBN/title-based book lookup, progress push, and note sync - Add HardcoverSyncMapStore for persistent local mapping of note IDs to Hardcover journal IDs - Add GraphQL queries/mutations for Hardcover API (insert/update/recreate journal entries) - Add DB migration for hardcover_note_mappings table (schema: hardcover-sync) - Add HardcoverSettings dialog component (connect/disconnect/enable toggle) - Add useHardcoverSync hook wired in Annotator for push-notes and push-progress events - Add Hardcover Sync section in BookMenu with per-book toggle (persisted in BookConfig) - Add HardcoverSettings type and DEFAULT_HARDCOVER_SETTINGS to system settings - Add hardcoverSyncEnabled per-book flag to BookConfig - Mount HardcoverSettingsWindow in Reader alongside KOSync and Readwise windows Sync is one-way (Readest → Hardcover), manual-action only, per-book opt-in. Supports idempotent note sync: insert new, skip unchanged, update changed, recreate on stale remote ID. * fix(hardcover): proxy API calls server-side to fix CORS, normalize Bearer token - Add /api/hardcover/graphql Next.js route that proxies POST requests server-side, bypassing CORS restrictions (Hardcover API has no Access-Control-Allow-Origin header) - On Tauri (desktop), calls Hardcover directly; on web, routes through the proxy - Normalize token in HardcoverClient: accept raw JWT or 'Bearer <jwt>' format - Update helper text to point to hardcover.app → Settings → API * fix(hardcover): surface note-sync no-ops and harden book resolution - Show toast feedback when book data is still loading, Hardcover is not configured, or the current book has no annotations/excerpts to sync - Show an explicit info toast when note sync finds no new changes - Parse Hardcover search results in the current hits/document response shape - Resolve note sync through ensureBookInLibrary for parity with progress sync - Add console logging for note/progress sync failures * debug(hardcover): add runtime instrumentation for note sync * feat(metadata): keep identifier stable and store ISBN separately - Add dedicated metadata.isbn field - Expose ISBN as its own editable field in book metadata - Preserve identifier semantics for existing source IDs and hashes - Route metadata auto-retrieval ISBN handling through the new field - Prefer metadata.isbn for Hardcover matching * fix(hardcover): avoid wasm sqlite for note mappings on web - Store Hardcover note sync mappings in localStorage on web - Keep sqlite-backed mappings for desktop/native environments - Remove the web-only database dependency from manual note sync * fix(hardcover): dedupe notes by payload hash across unstable note IDs - Add payload-hash lookup in HardcoverSyncMapStore - Reuse existing journal mapping when payload already synced - Prevent duplicate insertions when note IDs change or duplicate locally * fix(hardcover): avoid duplicate quote export when annotation note exists - Detect excerpt+annotation pairs for the same highlight - Skip standalone excerpt export when annotation has note text - Keep annotation export as the single source of truth * docs(hardcover): add consolidated change summary for review * fix(hardcover): suppress excerpt export by CFI when annotation note exists * fix(hardcover): suppress empty-note annotation duplicates when note exists * fix(hardcover): deduplicate notes by text and cfi base node * refactor(hardcover): optimize sync performance, add rate limiting and clean up debug tools * chore: remove dev-only change log from main * test(hardcover): add unit tests for sync mapping and client logic * chore: custom deployment and UI fixes * fix(hardcover): use timestamptz for accurate annotation time * fix(hardcover): use date scalar and RFC 3339 formatting for journal entries * Revert "chore: custom deployment and UI fixes" This reverts commit 0329aba7129d1e1ebf2c663804b8fba9a9f87b91. * Fix hardcover progress dates and surface imported ISBNs * Fix Hardcover currently reading sync * fix(hardcover): avoid promoting note sync status * style(hardcover): apply prettier formatting * test(hardcover): fix strict TypeScript assertions * test(hardcover): harden sync regression coverage * test(hardcover): fix lint and formatting regressions * fix(hardcover): narrow note dedupe range matching * refactor(hardcover): extract note dedupe helpers * refactor(isbn): extract metadata normalization helpers * feat(hardcover): improve synced quote formatting |
||
|
Huang Xin
|
956c71cd7b |
chore: migrate from ESLint to Biome for linting (#3694)
Replace ESLint with Biome for ~14x faster linting (~360ms vs ~5s). - Add biome.json with rules matching ESLint parity (Next.js, a11y, TypeScript, unused vars/imports) - Remove eslint, @typescript-eslint/*, eslint-config-next, eslint-plugin-jsx-a11y, @eslint/* from deps - Remove eslint.config.mjs - Update lint script to: tsgo --noEmit && biome check . - Fix 11 real code issues caught by Biome (banned types, explicit any, unsafe finally, unreachable code, shadowed names) - Disable Biome formatter (Prettier stays for Tailwind class sorting) Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
8850e6c00f |
feat(pdf): support TTS and annotation on PDFs, closes #2149 & #3462 (#3493)
* chore: bump jsdom to the latest version * feat(pdf): support TTS and annotation on PDFs, closes #2149 & closes #3462 |
||
|
Huang Xin
|
dac4f2e8ec | chore: experimental vinext build (#3486) | ||
|
JustADeer
|
b99c1bc19a |
feat(ui): image viewing mode support (#3328)
* feat(ui): image viewing mode support * Revert foliate-js submodule pointer to 72fda6a (CI-compatible) * Fix long-press image viewing in foliateViewer instead of foliate-js and other refactors * feat: add images navigation buttons and table viewer --------- Co-authored-by: Huang Xin <chrox.huang@gmail.com> |
||
|
Huang Xin
|
ea15906acf | fix(txt): fixed TXT import on platforms that CompressionStream is not working, closes #3255 (#3302) | ||
|
Huang Xin
|
411f9e236d | feat(metadata): support parsing series info from calibre exported EPUBs, closes #3259 (#3267) | ||
|
Huang Xin
|
9834bd57cf | feat(toc): add page number for nested TOC items, closes #2953 (#3174) | ||
|
Huang Xin
|
b89171a4d8 | compat(iOS): disable native CompressionStream/DecompressionStream API in zip.js for iOS 15.x (#3170) | ||
|
Huang Xin
|
9cd88fe839 | fix(progress): show correct progress info for subsections in FB2, closes #3136 (#3145) | ||
|
Mohammed Efaz
|
5bbc5ceccc |
feat(ai): AI reading assistant phase 2 (#3023)
* feat(ai): add dependencies * chore: bump zod version to default version * feat(ai): define types and model constants * feat(ai): ollama provider for local LLM * feat(ai): implement openrouter provider for cloud models * feat(settings): register ai settings panel in global dialog * refactor(ai): expose provider factory and service layer entry point * test(ai): add unit tests for the providers * test(ai): add unit tests for the providers * feat(ai): settings panel for ai configurations * refactor(ai): rewrite aipanel with autosave and greyed out disabled state * fix: remove unused onClose prop from aipanel * test(ai): update mock data * refactor(ai): remove models * refactor: use centralised defaults in system defaults * chore(ai): remove comments * fix(ai): merge default ai settings on load to prevent undefined values * refactor(ai): rewrite settings panel with autosave and model input * feat(ai): add ai tab with simplified highlighting * feat(sidebar): render AIAssistant for ai tab * feat(ai): add chat UI * feat(ai); add chat service with RAG context * feat(ai): temp debug logger * feat(ai): add RAG service * feat(ai): add text chunking utility * feat(ai): add structured method * feat(ai): add chatstructured method * feat(ai): add rag types nd structured output schema * feat(ai): add aistore, indexdb, bm25 * fix: update lock file * feat(ai): update types for AI SDK v5 * feat(ai): add placeholder gateway model constants * refactor(ai): update OllamaProvider for AI SDK * feat(ai): add native gateway provider * refactor(ai): update provider exports * refactor(ai): use streamText from AI SDK * refactor(ai): use embed from AI sdk * refactor(ai): update provider factory exports * feat(ai): add AI Elements and shadcn components * config: add shadcn component config * deps: add AI SDK and AI Elements dependencies * config: add ai packages to transpilePackages * refactor(ai): remove OpenRouterProvider and old tests * feat(ai):add assistant-ui components * feat(ai): add TauriChatAdapter for assistant-ui runtime * refactor(ai): remove ai-elements components * dep(ai): install assistant-ui and update next config * chore(ai): export adapters from service index * feat(ui): enhance ui components for assistant integration * feat(settings): migrate ai settings to gateway * feat(sidebar): integrate assistant-ui * feat: add ai settings toggle to sidebar content * feat: conditionally show ai tab in sidebar navigation * feat: update ai model constants for cheaper options * feat: add gateway provider with proxied embedding * feat: add timeouts to ollama provider health checks * feat: add retry logic to rag service embeddings * feat: add error recovery to ai store * feat: add ai feature tests * feat: add ai api endpoints * feat: add proxied gateway embedding provider * feat: add ai runtime utilities * feat: add ai retry utilities * feat: add tauri env example template * feat: add web env example template * chore: add env * feat(ai): update models and pricing, remove GLM-4.7-FlashX * feat(ai): improve system prompt with official headings and no numeric citations * feat(ai): optimize system prompt for tauri chat * feat(ui): refine ai chat UI and relocate sources * feat(ui): update ai settings panel with model pricing and custom model support * feat(ai): add custom model support to ai settings * test(ai): update constants tests for removed model * feat(api): implement ai chat proxy route * feat(api): implement ai embedding proxy route * feat(ai): implement ai gateway health check and proxy logic * feat(ai): simplify proxied embedding provider * feat(ui): improve markdown text rendering * feat(ui): add input group component * test(ai): update ai provider tests * feat(ai): add pageNumber to text chunk schema * feat(ai): implement page-based chunking with 1500 char formula * feat(ai): bump db to v2 and add store reset migration * feat(ai): transition rag pipeline to page level spoiler filtering * feat(ai): overhaul readest persona and antijailbreak prompt * feat(ai): update tauri adapter for page tracking and persona * chore(ai): export aiStore and logger from core index * feat(reader): integrate page tracking and manual index reset * feat(ui): add re-index action and reset logic to chat * chore: sync pnpm lockfile with ai dependencies * feat(utils): add browser-safe file utilities for web builds * refactor(utils): use dynamic tauri fs import to prevent web crashes * refactor(services): defer osType call to init() for web compatibility * refactor(services): import RemoteFile from file.web * refactor(services): import ClosableFile from file.web * fix(libs): cast Entry to any for getData access * fix(annotator): cast Overlayer to any for bubble access * refactor(ai): replace SparklesIcon with BookOpenIcon for index prompt * test(ai): add pageNumber to TextChunk mocks * test(ai): fix chunkSection signature in tests * chore: update files * fix(ai): prevent useLocalRuntime crash when adapter is null * refactor: optimize annotator overlay drawing * feat: stabilize AI assistant runtime and adapter * refactor: improve document zip loader type safety * feat: update tauri chat adapter for dynamic options * fix: restore architecture comments and refine platform properties * build: update lockfile with assistant-ui patch * fix(library): patch @assistant-ui/react for runtime initialization * build: update dependencies in readest-app * build: update root dependencies and patch configuration * fix(ai): patch @assistant-ui/react for thread deletion and runtime init * fix(ai): update assistant-ui patch with dist guards and deletion fallback * build: sync lockfile with assistant-ui patch updates * chore(env): update .gitignore by removing .env files from it * chore(env): update .gitignore by adding .env.local * chore(env): update .gitignore by adding .env*.local * fix: restore static osType import * chore: sync submodules with upstream/main * refactor: remove redundant file.web module and revert import * chore: update pnpm-lock.yaml * refactor: revert guards * refactor; remove deprecated codes and extract prompts.ts * refactor(ai): remove unused ragservice exports * refactor: remove unused ollama and embedding models * refactor: remove unused type * test: remove test for the now deleted constants * refactor: remove unused export * style: fix ui component formatting * style: fix core and style file formatting * test: fix broken ai provider import * fix: typescript error * fix: add eslint disable command * fix(deps): remove unused ai sdk provider util after v6 ai sdk migration * fix(patch): add lookbehind regex patch * feat(dep): upgrade vercel ai sdk to v6 and ai-sdk-ollama to v3 * chore: update lockfile for vercel ai sdk v6 * refactor(ai): remove EmbeddingModel generic for ai sdk v6 * refactor(ai): remove EmbeddingModel generic for ai sdk v6 * test(ai): update mock to use embeddingModel * fix(patch): add lookbehind regex patch for email autolinks in markdown * refactor(ai): use ai sdk v6 syntax * fix: prettier formatting * chore: revert cargo.lock * fix(ai): update proxied embedding model to v3 spec * feat(ai): add aiconversation types for chat persistence * feat(ai): add conversation/message indexeddb and crud operations * feat(ai): create aiChatStore zustand store for chat state management * feat(notebook): add notebookactivetab state for Notes/AI * refactor(ai): refine conversation and message types for persistence * feat(types): add notebookActiveTab to ReadSettings type * chore: update deps * feat: add notebookactive tab default value * feat: add hook for ai chat * feat: update left side panel with history/chat icon * feat: integrate ChatHistoryView into sidebar content * feat: create UI for managing AI chat history * feat: implement persistent history with assistant-ui adapter * feat: create tab navigation component for notes and AI * feat: add tab navigation and AI assistant view * feat: update header to display active tab title * fix: formatting * feat: remove title and update new chat button * fix: formatting * fix: revert tooltip and styling * feat: implement cross-platform ask dialog bridge * feat(ai): preserve history during ui clear & use native dialogs * fix: align notebook navigation height with sidebar tabs * fix(ai): add missing dependency to handleDeleteConversation hook * docs: update PROJECT.md with session highlights * chore: delete projectmd * chore: update package.json and lock file * chore: update package.json * chore: remove patch * chore: upgrade react types to 19 and show ai features only in development mode for now --------- Co-authored-by: Huang Xin <chrox.huang@gmail.com> |
||
|
xijibomi-coffee
|
f875ba88ac |
perf: use native walkdir for recursive imports from directory (#2993)
* fix(perf): replace JS recursion with native Rust walkdir for imports * fix: implement security scope check in rust recursive scanner * refactor and format code --------- Co-authored-by: Huang Xin <chrox.huang@gmail.com> |
||
|
Huang Xin
|
d9a6cffe78 | feat(discord): support show reading status with Discord Rich Presence, closes #1538 (#2998) | ||
|
Huang Xin
|
2d7d6b08a9 | compat(opds): parse attachment filename from download requests, closes #2969 (#2976) |