forked from akai/readest
4874eb9ae7e19e457ad64f8dceb5b2a49b2b301e
12 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Huang Xin
|
4874eb9ae7 | feat(reader): add TTS highlight granularity setting (word or sentence) (#4807) | ||
|
Huang Xin
|
b87c735c1e |
fix(tts): keep native System TTS reading past unspeakable chunks offline (#4613, #4408) (#4716)
Android System TTS (and iOS) read-aloud could stop offline and refuse to continue — #4613 "stops at the end of the chapter, won't advance" and #4408 "stops at random intervals" — after which the play/headphone controls felt wedged. Root cause: `TTSController.#speak` only auto-advances when the last event code is `end`. The native client surfaces an offline engine failure as a terminal `error` code (Android `UtteranceProgressListener.onError`). This typically happens on a specific utterance the offline engine can't synthesize — e.g. an unsupported character — characteristically the first utterance after a chapter boundary, even with a local/offline voice (online, engines often fall back to network synthesis, which is why it only breaks offline). On `error` the controller never called `forward()` and left `state` stuck at `playing`, so playback dead-ended and the controls couldn't recover. Edge/Web clients throw instead (handled by `error()`), so only the native client hit this. Fix (native-scoped, no change to the Edge/Web path): when the active client is the native client and an utterance ends with a terminal `error` (still playing, not aborted, not one-time), skip that chunk and advance just as a normal `end` would — re-speaking the same unsynthesizable text would only fail again. A consecutive-error cap stops playback gracefully if the engine can't speak anything, so a wholly-unusable engine doesn't silently race to the end of the book and the state machine always leaves `playing`. Tests: tts-controller covers skip-on-error advancing past a bad chunk, and the consecutive-error cap stopping gracefully (bounded, not wedged in playing). Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
96d65d9960 |
feat(tts): add native local iOS TTS (AVSpeechSynthesizer) (#4697)
Implement on-device iOS text-to-speech using AVSpeechSynthesizer, mirroring the Android native TextToSpeech plugin so the shared NativeTTSClient drives both platforms through the same command and tts_events contract. - Swift NativeTTSPlugin: speak/stop/pause/resume/rate/pitch/voice and voice enumeration, with region-disambiguated duplicate voice names and a small preUtteranceDelay to avoid first-word clipping. - Enable the native TTS client on iOS in TTSController. - Make TTS teardown resilient: reset UI state up front and tear down the controller, media session, and background audio in parallel so a slow native shutdown can never leave the TTS icon or lock-screen session stuck on. - Keep iOS on navigator.mediaSession for the lock screen (Android uses the native foreground service), which restores the Edge TTS cover and current-sentence metadata. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
86f5502724 |
fix: bot-review robustness fixes (TTS sync, updater, nightly, a11y) (#4659)
Cherry-picked and re-verified the applicable subset of julianshen/readest@fa1b74a0 (its "address PR #15 bot reviews" commit). The fork-only AI-annotation-tool change was dropped — readest has no 'ai' toolbar tool. Each logic fix is covered by a failing-first test. - TTS position sequence is now an app-wide monotonic counter, so a fresh TTSController (constructed per `tts-speak`) isn't dropped by consumers holding `lastSequenceSeen` from a prior session. - share.ts only swallows AbortError (user cancel); other failures — e.g. NotAllowedError when a quick action fires without a user gesture — fall back to the clipboard so the text still reaches the user. - document.isTxt tolerates MIME params (text/plain;charset=utf-8), uppercase extensions (BOOK.TXT), and a nameless Blob, so a TXT can't slip onto the non-text path and yield a null book. - updater getNightlyPlatformKey matches x86_64/aarch64 explicitly; a 32-bit or otherwise unknown arch yields no nightly instead of mis-routing to aarch64. - UpdaterWindow downloadWithProgress resolves on tauriDownload completion even when Content-Length is absent (no more hang on portable/AppImage/Android). - nightly_update.rs uses async tokio::fs::read in the async command. - nightly.yml: serialize runs via a concurrency group (no cancel) and persist-credentials:false on checkouts. - edge TTS route only emits the word-boundary header when it fits under ~8KB; oversized values get dropped by proxies, and the client falls back to []. - RSVPOverlay drops the contradictory aria-disabled on the functional rate button (it opens the pace picker). - nightly verify harness handles artifact stream errors instead of crashing. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
e327d0c992 |
feat(tts): reuse the speaking session across paragraph & RSVP modes (#4657)
Switching into Paragraph or RSVP mode while TTS is already playing now syncs to the live session instead of forcing a stop + restart inside the mode. Bundles several related TTS fixes uncovered along the way. Session reuse (enter from normal mode): - TTSController.redispatchPosition() re-emits the current position on the canonical tts-position signal with a fresh sequence. - useTTSControl answers a new tts-sync-request by replaying the current position then playback state (position-first so RSVP's paused handler can't discard it). - Paragraph & RSVP engage following on entry and dispatch the request; no-op when no session exists. RSVP refinements: - Reusing a session skips the start dialog and the get-ready countdown (starts externally driven); gated on a live tts-playback-state signal so the countdown can't flash. - Stopping TTS now pauses RSVP instead of resuming its own pacing. Word-sync fixes: - rangeTextExcludingInert honours the range offsets inside a single text node, fixing word-highlight drift on middle sentences of single-<span> paragraphs (Edge word highlighting). - foliate-js TTS.from() starts at the sentence containing the selection, not the next one (submodule bump). - Selecting a word and starting TTS now clears the selection. - Dev-only [TTS] word-sync trace (stripped from production builds). Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
bfb85c2f68 |
feat(reader): sync paragraph mode & speed reader with TTS read-along (#3235) (#4576)
* docs(reader): TTS-sync design spec for paragraph mode + RSVP (#3235) Hardened via brainstorming + /autoplan (CEO/Design/Eng dual-voice review). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(tts): emit canonical tts-position event from TTSController (#3235) Controller emits { cfi, kind, sectionIndex, sequence } alongside the existing tts-highlight-mark/-word events. Monotonic sequence lets downstream consumers (paragraph mode, RSVP — later slices) drop out-of-order positions. Additive; existing events untouched. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(reader): containment+cursor CFI->index mappers for TTS sync (#3235) RSVPController.syncToCfi + setExternallyDriven: containment match (fixes mid-token skip), monotonic cursor + binary search (avoids O(N)-per-word jank, no per-word getCFI), -1/no-op on no match (no silent jump to word 0), timer suspension while externally driven. ParagraphIterator.findIndexByRange: hinted + binary-search containment mapper returning -1 on no match (never first()). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(tts): forward tts-position + tts-playback-state onto the app bus (#3235) useTTSControl republishes the controller's canonical tts-position (tagged with bookKey) via a dedicated listener — NOT inside the suppression-gated highlight handlers, so page-follow suppression can't silently desync the modes. Adds tts-playback-state (playing/paused/stopped) so RSVP can track playback without the hook-local isPlaying. Verified by extending the real-foliate-view browser harness. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(reader): paragraph mode follows TTS playback (#3235) When paragraph mode + TTS are both active, the focused paragraph follows the spoken position (sentence granularity, all engines). Section-generation contract (stash cross-section position, apply after the iterator re-inits); sync-focus path that does NOT arm isFocusingRef (avoids the relocate-eaten wrong-section paragraph-0 bug); stale-sequence drop; decouple on manual nav, re-engage on next playing. Start-alignment + visible indicator deferred to later slices. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(rsvp): speed reader follows TTS playback (#3235) Edge word-boundary voices: RSVP shows the spoken word via syncToCfi. Non-Edge (sentence-only) voices: sentence-paced estimator (clamp 60..600 wpm from voice rate, hold at +60 words cap, snap to first word on each new sentence mark). RSVP auto-advance suspended while TTS-driven. Decouple on manual nav via a rsvp-manual-nav signal; re-engage on next playing. Cross-section positions re-extract then apply. Pure decideRsvpTtsPosition helper unit-tested. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(reader): fixed-layout gate + ttsSyncStatus for TTS sync (#3235) Gate sync to reflowable books (D7): fixed-layout reports 'unsupported' and never engages. Both modes expose ttsSyncStatus (idle/following/syncing/ decoupled/unsupported) as the data source for the upcoming indicator. RSVPControl now forwardRef-exposes the status via an imperative handle. Cross-bookKey events ignored (regression-tested). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(reader): 'following audio' indicator for TTS sync (#3235) 5-state pill (following/syncing/decoupled, idle+unsupported render null) shown top-center in the paragraph overlay and as a status row in the RSVP overlay. Decoupled state is the tap-to-resume control; first decouple fires a one-time toast. eink-bordered, glyph+text (no color-only), RTL logical props, touch targets, safe-area top inset. RSVP 'plain' variant matches its themed surface; non-Edge shows '· estimated'. New i18n keys need extraction before merge. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(rsvp): in-overlay TTS toggle + audio-paced speed control (#3235) Voice-glyph audio toggle in the RSVP control row (trailing, by the gear) starts/ stops read-along from inside the full-screen overlay, start-aligned to the current word (range validated against the live doc). While TTS-driven, the WPM control shows a locked 'Audio pace' affordance that opens a compact rate picker; rate changes go through a new tts-set-rate bus event reusing the existing throttled setRate path. Pure buildRsvpTtsSpeakDetail helper unit-tested. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(reader): e2e paragraph mode follows TTS across a section boundary (#3235) Real <foliate-view> browser e2e: with paragraph mode active, the focused paragraph follows the TTS walk and re-targets to the new section after a Ch4->Ch5 boundary (proves no stuck wrong-section paragraph-0 / isFocusingRef trap). Asserts on the owning section of the current range. Test-only. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * i18n(reader): translate TTS-sync strings across 33 locales (#3235) Following audio / · estimated / Resume audio / Stopped following audio / Play audio / Pause audio / Audio pace / Speed follows audio. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(reader): resolve TTS CFI anchors across iframe realms (#3235) RSVP and paragraph follow silently failed to track the spoken word: the CFI anchor from view.resolveCFI(...).anchor(doc) is a Range created in the book iframe's realm, so 'anchor instanceof Range' (top realm) was always false (cross-realm instanceof) -> resolveCfiToRange/applySyncCfi returned null -> syncToCfi never advanced. Add isRangeLike() duck-type (cloneRange is unique to Range) and use it at all 4 CFI-resolution sites. Confirmed live via CDP: before = syncToCfi false (frozen); after = exact word map + RSVP follows Edge TTS at ~171 wpm (audio pace). Unit tests reproduce the cross-realm anchor (jsdom is single-realm so the old code passed there but died in the app). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(rsvp): stop estimator/word fight + map transport to TTS play/pause (#3235) Two read-along refinements (verified live via CDP with Edge TTS): 1. No more jump-ahead-then-snap-back flashing. Word-boundary engines (Edge) emit BOTH sentence marks and word boundaries; RSVP was routing sentence -> the estimator (self-paces ~190xrate, up to +60 words ahead) while word positions snapped it back. Now once a word position is seen, sentence positions are ignored and any running estimator is stopped, so words alone drive RSVP. 2. The RSVP transport (center play/pause, Space, center-tap) maps to TTS play/pause while read-along is engaged (tts-toggle-play), instead of RSVP's own suspended timer. Pausing TTS keeps RSVP suspended (no runaway); a full stop releases it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(rsvp): keep indicator on pause + reach dict management from RSVP (#3235) - Pausing read-along no longer dismisses the 'following audio' indicator / 'Audio pace' lock (layout shift). New 'paused' sync status keeps the indicator row and WPM lock present while TTS is engaged-but-paused; only a full stop clears them. Verified live via CDP: pause keeps the layout, no shift. - Dict management is reachable from RSVP: the settings dialog is z-50, far below the full-screen RSVP overlay (z-[10000]), so it opened invisibly behind it. handleManageDictionary now exits RSVP first (position saved/resumable) so management shows over the reader. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(rsvp): show dict management over RSVP instead of exiting it (#3235) Per feedback: opening dictionary management from the RSVP lookup popup no longer closes the speed reader. The settings dialog is raised above the full-screen RSVP overlay (z-[10000] -> SettingsDialog !z-[10050]) so it shows on top, and RSVP's capture-phase keyboard handler bails while the settings dialog is open so its inputs accept Space and Escape closes settings (not RSVP). Verified live via CDP: management opens over RSVP, RSVP stays active behind it, Escape returns to it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(dict): only apply drag-handle margin compensation when the handle shows (#3235) The dictionary sheet header used -mt-4 to compensate for Dialog's drag handle, but that handle is sm:hidden (shown only below sm). On sm+ the handle is display:none, so -mt-4 pulled the header up into the top edge (broken layout when the lookup renders as a sheet on a short/wide window). Mirror the handle's breakpoint: -mt-4 sm:mt-0. Verified live via CDP. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(reader): in-mode TTS audio toggle for paragraph mode (#3235) Paragraph mode already follows TTS, but there was no way to start read-along from inside it. Add an audio toggle to the ParagraphBar (mirroring RSVP's): it starts TTS start-aligned to the focused paragraph (range validated live, + section index) and stops it. Track session-active vs playing so a pause keeps the indicator ('paused' status) instead of collapsing to idle. Pure buildParagraphTtsSpeakDetail helper unit-tested. Verified live via CDP: tapping the icon starts audio from the focused paragraph and the focus follows speech. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(reader): highlight current TTS word/sentence in paragraph mode (#3235) Paragraph mode follows TTS by advancing the focused paragraph, but the spoken word wasn't highlighted within it like normal mode. The overlay renders a CLONE of the paragraph, so the iframe's TTS highlight isn't visible there — reproduce it on the clone with the CSS Custom Highlight API (no DOM mutation, spans inline boundaries natively, leaves the fade-in animation untouched). - TTSController already tags tts-position with kind word|sentence. The hook decides granularity: word boundaries (Edge) drive a per-word highlight; once seen, the coarse sentence event is skipped so the whole sentence doesn't flicker over the current word. Engines without word boundaries (WebSpeech/Native) fall back to the sentence highlight. - Offsets are computed relative to the paragraph start (so they map 1:1 onto the clone's text) and tagged with the paragraph index so a stale highlight never paints the wrong paragraph. Cleared on stop / section change / disabled. - The ::highlight() style mirrors the user's ttsHighlightOptions color+style. Pure helpers (offset math, word/sentence decision, css builder) unit-tested. Verified live via CDP: word highlight tracks Edge word-by-word and follows across paragraph boundaries (news -> ... -> ladies), matching the TTS color. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
a56cc6c61a |
feat(tts): word-by-word highlighting for Edge TTS, closes #4017 (#4566)
Highlight each word as it is spoken (Edge TTS only) instead of keeping the whole sentence highlighted, and keep the view tracking the spoken word across page boundaries. Word boundaries - Capture Edge's audio.metadata WordBoundary frames (offset/duration in 100ns ticks plus the verbatim input-text span) in the Tauri, browser, and Cloudflare-Workers WebSocket transports. - Carry boundaries through the authenticated HTTPS proxy route via an X-TTS-Word-Boundaries response header (percent-encoded JSON, ASCII-safe), so word highlighting works on the web where the browser cannot open the wss connection directly. Cache them alongside the audio blob URL. Highlighting - Sync a requestAnimationFrame loop to audio.currentTime against the boundary table and highlight the word sub-range within the spoken sentence. Synthesis stays sentence-level (natural prosody); only the visual highlight is word-level. - Suppress the sentence highlight when the active client reports word boundaries and draw the first word immediately, so the whole sentence never flashes before the first word. Fall back to the sentence highlight when a chunk has no boundaries (other engines, empty metadata). - Re-apply the current word (not the sentence) when the view relocates. Page following - Turn the page as soon as the spoken word crosses a page boundary (a tts-highlight-word event scrolls only when the word is outside the visible range), instead of waiting for the next sentence. - Check the word's position for the "back to TTS location" badge so it no longer appears while the view follows the word onto the next page. Also fixes a pre-existing bug where the browser WebSocket was constructed with an options object (valid only for the Node ws package), which threw in browsers and made the wss path unusable on the web. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Huang Xin
|
ef603852b7 |
feat(tts): hotkey to highlight the currently-spoken sentence (#4085) (#4368)
Add a "Highlight Current Sentence" keyboard action (default Shift+M, in the Text to Speech shortcut section) that persists the sentence TTS is reading aloud as a normal highlight using the user's default style/color — no text selection, eyes-off, silent, and idempotent (a repeat press on the same sentence is a no-op rather than a duplicate). Flow: the shortcut handler in useBookShortcuts dispatches tts-highlight-sentence → useTTSControl (which owns the TTSController) resolves the current sentence via the new TTSController.getSpokenSentence() and relays create-tts-highlight → Annotator builds the BookNote with the pure, unit-tested buildTTSSentenceHighlight helper and persists/renders it like any other highlight. Closes #4085 Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
ad55375f89 |
fix(tts): preserve state on AbortError to keep rate changes effective on iOS, closes #3949 (#3988)
iOS WKWebView's in-flight audio.play() rejects with AbortError after audio.src is reset during a stop+restart cycle. That rejection leaks through one of the .catch chains into TTSController.error(), which unconditionally flipped state to 'stopped' — desyncing the state machine so subsequent rate changes fell into the no-op else branch and #speak's auto-forward gate stopped firing at paragraph end. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
9c273d79fc | fix(tts): fixed race condition on pause/resume, closes #3825 (#3905) | ||
|
Huang Xin
|
21795e5cd3 |
fix(tts): avoid race condition in preloadNextSSML causing wrong highlights (#3748)
preloadNextSSML() called tts.next() interleaved with await, creating async gaps where the concurrent #speak() could dispatch marks against corrupted #ranges state (replaced by next() for a different block). Since mark names restart at "0" per block, marks resolved to the wrong text position, causing accidental page turns and highlights far from the current sentence. Fix: gather all tts.next() and tts.prev() calls synchronously (no await between them) so no async code can see the intermediate #ranges state. Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
0e516f6e56 |
chore(test): add unit tests and enforce dash-case naming for test files (#3715)
Add ~290 new unit tests covering utils (lru, diff, css, a11y, walk, usage, txt-worker), services (EdgeTTSClient, transformers), and suppress noisy console output across all test files. Rename 27 camelCase test files to dash-case for consistency. Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |