forked from akai/readest
* docs(reader): TTS-sync design spec for paragraph mode + RSVP (#3235) Hardened via brainstorming + /autoplan (CEO/Design/Eng dual-voice review). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(tts): emit canonical tts-position event from TTSController (#3235) Controller emits { cfi, kind, sectionIndex, sequence } alongside the existing tts-highlight-mark/-word events. Monotonic sequence lets downstream consumers (paragraph mode, RSVP — later slices) drop out-of-order positions. Additive; existing events untouched. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(reader): containment+cursor CFI->index mappers for TTS sync (#3235) RSVPController.syncToCfi + setExternallyDriven: containment match (fixes mid-token skip), monotonic cursor + binary search (avoids O(N)-per-word jank, no per-word getCFI), -1/no-op on no match (no silent jump to word 0), timer suspension while externally driven. ParagraphIterator.findIndexByRange: hinted + binary-search containment mapper returning -1 on no match (never first()). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(tts): forward tts-position + tts-playback-state onto the app bus (#3235) useTTSControl republishes the controller's canonical tts-position (tagged with bookKey) via a dedicated listener — NOT inside the suppression-gated highlight handlers, so page-follow suppression can't silently desync the modes. Adds tts-playback-state (playing/paused/stopped) so RSVP can track playback without the hook-local isPlaying. Verified by extending the real-foliate-view browser harness. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(reader): paragraph mode follows TTS playback (#3235) When paragraph mode + TTS are both active, the focused paragraph follows the spoken position (sentence granularity, all engines). Section-generation contract (stash cross-section position, apply after the iterator re-inits); sync-focus path that does NOT arm isFocusingRef (avoids the relocate-eaten wrong-section paragraph-0 bug); stale-sequence drop; decouple on manual nav, re-engage on next playing. Start-alignment + visible indicator deferred to later slices. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(rsvp): speed reader follows TTS playback (#3235) Edge word-boundary voices: RSVP shows the spoken word via syncToCfi. Non-Edge (sentence-only) voices: sentence-paced estimator (clamp 60..600 wpm from voice rate, hold at +60 words cap, snap to first word on each new sentence mark). RSVP auto-advance suspended while TTS-driven. Decouple on manual nav via a rsvp-manual-nav signal; re-engage on next playing. Cross-section positions re-extract then apply. Pure decideRsvpTtsPosition helper unit-tested. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(reader): fixed-layout gate + ttsSyncStatus for TTS sync (#3235) Gate sync to reflowable books (D7): fixed-layout reports 'unsupported' and never engages. Both modes expose ttsSyncStatus (idle/following/syncing/ decoupled/unsupported) as the data source for the upcoming indicator. RSVPControl now forwardRef-exposes the status via an imperative handle. Cross-bookKey events ignored (regression-tested). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(reader): 'following audio' indicator for TTS sync (#3235) 5-state pill (following/syncing/decoupled, idle+unsupported render null) shown top-center in the paragraph overlay and as a status row in the RSVP overlay. Decoupled state is the tap-to-resume control; first decouple fires a one-time toast. eink-bordered, glyph+text (no color-only), RTL logical props, touch targets, safe-area top inset. RSVP 'plain' variant matches its themed surface; non-Edge shows '· estimated'. New i18n keys need extraction before merge. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(rsvp): in-overlay TTS toggle + audio-paced speed control (#3235) Voice-glyph audio toggle in the RSVP control row (trailing, by the gear) starts/ stops read-along from inside the full-screen overlay, start-aligned to the current word (range validated against the live doc). While TTS-driven, the WPM control shows a locked 'Audio pace' affordance that opens a compact rate picker; rate changes go through a new tts-set-rate bus event reusing the existing throttled setRate path. Pure buildRsvpTtsSpeakDetail helper unit-tested. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(reader): e2e paragraph mode follows TTS across a section boundary (#3235) Real <foliate-view> browser e2e: with paragraph mode active, the focused paragraph follows the TTS walk and re-targets to the new section after a Ch4->Ch5 boundary (proves no stuck wrong-section paragraph-0 / isFocusingRef trap). Asserts on the owning section of the current range. Test-only. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * i18n(reader): translate TTS-sync strings across 33 locales (#3235) Following audio / · estimated / Resume audio / Stopped following audio / Play audio / Pause audio / Audio pace / Speed follows audio. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(reader): resolve TTS CFI anchors across iframe realms (#3235) RSVP and paragraph follow silently failed to track the spoken word: the CFI anchor from view.resolveCFI(...).anchor(doc) is a Range created in the book iframe's realm, so 'anchor instanceof Range' (top realm) was always false (cross-realm instanceof) -> resolveCfiToRange/applySyncCfi returned null -> syncToCfi never advanced. Add isRangeLike() duck-type (cloneRange is unique to Range) and use it at all 4 CFI-resolution sites. Confirmed live via CDP: before = syncToCfi false (frozen); after = exact word map + RSVP follows Edge TTS at ~171 wpm (audio pace). Unit tests reproduce the cross-realm anchor (jsdom is single-realm so the old code passed there but died in the app). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(rsvp): stop estimator/word fight + map transport to TTS play/pause (#3235) Two read-along refinements (verified live via CDP with Edge TTS): 1. No more jump-ahead-then-snap-back flashing. Word-boundary engines (Edge) emit BOTH sentence marks and word boundaries; RSVP was routing sentence -> the estimator (self-paces ~190xrate, up to +60 words ahead) while word positions snapped it back. Now once a word position is seen, sentence positions are ignored and any running estimator is stopped, so words alone drive RSVP. 2. The RSVP transport (center play/pause, Space, center-tap) maps to TTS play/pause while read-along is engaged (tts-toggle-play), instead of RSVP's own suspended timer. Pausing TTS keeps RSVP suspended (no runaway); a full stop releases it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(rsvp): keep indicator on pause + reach dict management from RSVP (#3235) - Pausing read-along no longer dismisses the 'following audio' indicator / 'Audio pace' lock (layout shift). New 'paused' sync status keeps the indicator row and WPM lock present while TTS is engaged-but-paused; only a full stop clears them. Verified live via CDP: pause keeps the layout, no shift. - Dict management is reachable from RSVP: the settings dialog is z-50, far below the full-screen RSVP overlay (z-[10000]), so it opened invisibly behind it. handleManageDictionary now exits RSVP first (position saved/resumable) so management shows over the reader. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(rsvp): show dict management over RSVP instead of exiting it (#3235) Per feedback: opening dictionary management from the RSVP lookup popup no longer closes the speed reader. The settings dialog is raised above the full-screen RSVP overlay (z-[10000] -> SettingsDialog !z-[10050]) so it shows on top, and RSVP's capture-phase keyboard handler bails while the settings dialog is open so its inputs accept Space and Escape closes settings (not RSVP). Verified live via CDP: management opens over RSVP, RSVP stays active behind it, Escape returns to it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(dict): only apply drag-handle margin compensation when the handle shows (#3235) The dictionary sheet header used -mt-4 to compensate for Dialog's drag handle, but that handle is sm:hidden (shown only below sm). On sm+ the handle is display:none, so -mt-4 pulled the header up into the top edge (broken layout when the lookup renders as a sheet on a short/wide window). Mirror the handle's breakpoint: -mt-4 sm:mt-0. Verified live via CDP. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(reader): in-mode TTS audio toggle for paragraph mode (#3235) Paragraph mode already follows TTS, but there was no way to start read-along from inside it. Add an audio toggle to the ParagraphBar (mirroring RSVP's): it starts TTS start-aligned to the focused paragraph (range validated live, + section index) and stops it. Track session-active vs playing so a pause keeps the indicator ('paused' status) instead of collapsing to idle. Pure buildParagraphTtsSpeakDetail helper unit-tested. Verified live via CDP: tapping the icon starts audio from the focused paragraph and the focus follows speech. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(reader): highlight current TTS word/sentence in paragraph mode (#3235) Paragraph mode follows TTS by advancing the focused paragraph, but the spoken word wasn't highlighted within it like normal mode. The overlay renders a CLONE of the paragraph, so the iframe's TTS highlight isn't visible there — reproduce it on the clone with the CSS Custom Highlight API (no DOM mutation, spans inline boundaries natively, leaves the fade-in animation untouched). - TTSController already tags tts-position with kind word|sentence. The hook decides granularity: word boundaries (Edge) drive a per-word highlight; once seen, the coarse sentence event is skipped so the whole sentence doesn't flicker over the current word. Engines without word boundaries (WebSpeech/Native) fall back to the sentence highlight. - Offsets are computed relative to the paragraph start (so they map 1:1 onto the clone's text) and tagged with the paragraph index so a stale highlight never paints the wrong paragraph. Cleared on stop / section change / disabled. - The ::highlight() style mirrors the user's ttsHighlightOptions color+style. Pure helpers (offset math, word/sentence decision, css builder) unit-tested. Verified live via CDP: word highlight tracks Edge word-by-word and follows across paragraph boundaries (news -> ... -> ladies), matching the TTS color. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Huang Xin
@@ -1,5 +1,6 @@
|
||||
import { useCallback, useEffect, useRef, useState } from 'react';
|
||||
import { useReaderStore } from '@/store/readerStore';
|
||||
import { useBookDataStore } from '@/store/bookDataStore';
|
||||
import { useEnv } from '@/context/EnvContext';
|
||||
import { FoliateView } from '@/types/view';
|
||||
import { eventDispatcher } from '@/utils/event';
|
||||
@@ -7,12 +8,32 @@ import { saveViewSettings } from '@/helpers/settings';
|
||||
import { ParagraphIterator } from '@/utils/paragraph';
|
||||
import { getParagraphPresentation } from '@/utils/paragraphPresentation';
|
||||
import { DEFAULT_PARAGRAPH_MODE_CONFIG } from '@/services/constants';
|
||||
import { isRangeLike } from '@/utils/range';
|
||||
import {
|
||||
buildParagraphTtsSpeakDetail,
|
||||
computeParagraphHighlightOffsets,
|
||||
decideParagraphTtsHighlight,
|
||||
} from '@/app/reader/components/paragraph/paragraphTts';
|
||||
|
||||
interface UseParagraphModeProps {
|
||||
bookKey: string;
|
||||
viewRef: React.RefObject<FoliateView | null>;
|
||||
}
|
||||
|
||||
// Derived state for the TTS-sync indicator (later slice renders it).
|
||||
// - 'unsupported': fixed-layout book; sync can never engage.
|
||||
// - 'idle': TTS not playing / not engaged.
|
||||
// - 'following': actively following the spoken position.
|
||||
// - 'syncing': a cross-section position is pending a re-init.
|
||||
// - 'decoupled': was following, user took manual control (TTS still playing).
|
||||
export type TtsSyncStatus =
|
||||
| 'unsupported'
|
||||
| 'idle'
|
||||
| 'following'
|
||||
| 'syncing'
|
||||
| 'decoupled'
|
||||
| 'paused';
|
||||
|
||||
export interface ParagraphState {
|
||||
isActive: boolean;
|
||||
isLoading: boolean;
|
||||
@@ -24,6 +45,11 @@ export interface ParagraphState {
|
||||
export const useParagraphMode = ({ bookKey, viewRef }: UseParagraphModeProps) => {
|
||||
const { envConfig } = useEnv();
|
||||
const { getViewSettings, setViewSettings, getProgress } = useReaderStore();
|
||||
const { getBookData } = useBookDataStore();
|
||||
|
||||
// Fixed-layout gate (D7): paragraph mode must never engage TTS sync for a
|
||||
// fixed-layout book. Mirrors how other reader code reads the flag.
|
||||
const isFixedLayout = getBookData(bookKey)?.isFixedLayout ?? false;
|
||||
|
||||
const iteratorRef = useRef<ParagraphIterator | null>(null);
|
||||
const currentDocIndexRef = useRef<number | undefined>(undefined);
|
||||
@@ -41,6 +67,29 @@ export const useParagraphMode = ({ bookKey, viewRef }: UseParagraphModeProps) =>
|
||||
paragraphCfi: string;
|
||||
docIndex: number;
|
||||
} | null>(null);
|
||||
// TTS-sync (one-way follower): TTS is the clock, paragraph mode follows it.
|
||||
const followingTtsRef = useRef(false);
|
||||
const lastSequenceSeenRef = useRef(-Infinity);
|
||||
const pendingSyncRef = useRef<{
|
||||
cfi: string;
|
||||
sequence: number;
|
||||
sectionIndex: number;
|
||||
kind?: 'word' | 'sentence';
|
||||
} | null>(null);
|
||||
// Whether the active TTS session has emitted word boundaries (Edge). Drives the
|
||||
// word-vs-sentence highlight granularity; reset when a session fully stops.
|
||||
const hasWordPositionsRef = useRef(false);
|
||||
// Holds the latest applySyncCfi so initIterator can apply a pending cross-
|
||||
// section sync after re-init without a circular useCallback dependency.
|
||||
const applySyncCfiRef = useRef<((cfi: string, highlight: boolean) => void) | null>(null);
|
||||
// Latest TTS playback-state for this book ('playing' vs not), used to derive
|
||||
// the sync status alongside the follow/pending refs.
|
||||
const ttsPlayingRef = useRef(false);
|
||||
// A TTS session exists (playing OR paused) for this book. Distinct from
|
||||
// ttsPlayingRef so a pause keeps the indicator + audio toggle "active" (the
|
||||
// mode is still on); only a full stop clears it. Drives the bar's audio toggle.
|
||||
const ttsActiveRef = useRef(false);
|
||||
const refreshTtsSyncStatusRef = useRef<(() => void) | null>(null);
|
||||
bookKeyRef.current = bookKey;
|
||||
|
||||
const [paragraphState, setParagraphState] = useState<ParagraphState>({
|
||||
@@ -51,6 +100,33 @@ export const useParagraphMode = ({ bookKey, viewRef }: UseParagraphModeProps) =>
|
||||
currentRange: null,
|
||||
});
|
||||
|
||||
const [ttsSyncStatus, setTtsSyncStatus] = useState<TtsSyncStatus>(
|
||||
isFixedLayout ? 'unsupported' : 'idle',
|
||||
);
|
||||
|
||||
// Whether a TTS session is engaged (playing or paused) for this book. Drives
|
||||
// the bar's audio toggle active/idle glyph (#3235).
|
||||
const [ttsActive, setTtsActive] = useState(false);
|
||||
|
||||
// Derive the indicator status from the current refs + the latest playback
|
||||
// state and push it to React state so the indicator re-renders. Fixed-layout
|
||||
// always wins.
|
||||
const refreshTtsSyncStatus = useCallback(() => {
|
||||
setTtsSyncStatus(() => {
|
||||
if (isFixedLayout) return 'unsupported';
|
||||
// No session at all (never started / fully stopped): idle.
|
||||
if (!ttsActiveRef.current) return 'idle';
|
||||
// Engaged but paused: keep the indicator visible (mode still on).
|
||||
if (!ttsPlayingRef.current) return 'paused';
|
||||
if (!followingTtsRef.current) return 'decoupled';
|
||||
if (pendingSyncRef.current) return 'syncing';
|
||||
return 'following';
|
||||
});
|
||||
}, [isFixedLayout]);
|
||||
// Lets initIterator refresh the status after clearing a pending sync without
|
||||
// pulling refreshTtsSyncStatus into its (widely-depended-on) deps array.
|
||||
refreshTtsSyncStatusRef.current = refreshTtsSyncStatus;
|
||||
|
||||
const paragraphConfig = getViewSettings(bookKey)?.paragraphMode ?? DEFAULT_PARAGRAPH_MODE_CONFIG;
|
||||
|
||||
const getPrimaryContent = useCallback(() => {
|
||||
@@ -135,7 +211,8 @@ export const useParagraphMode = ({ bookKey, viewRef }: UseParagraphModeProps) =>
|
||||
const resolved = view.resolveCFI(progressLocation);
|
||||
if (!resolved || resolved.index !== docIndex) return null;
|
||||
const anchor = resolved.anchor(doc);
|
||||
if (anchor instanceof Range) return anchor;
|
||||
// Realm-safe: iframe-realm Range fails top-realm `instanceof Range`.
|
||||
if (isRangeLike(anchor)) return anchor;
|
||||
if (anchor) {
|
||||
const range = doc.createRange();
|
||||
range.selectNodeContents(anchor);
|
||||
@@ -155,7 +232,8 @@ export const useParagraphMode = ({ bookKey, viewRef }: UseParagraphModeProps) =>
|
||||
const resolved = view.resolveCFI(lastParagraph.paragraphCfi);
|
||||
if (!resolved || resolved.index !== docIndex) return null;
|
||||
const anchor = resolved.anchor(doc);
|
||||
if (anchor instanceof Range) return anchor;
|
||||
// Realm-safe: iframe-realm Range fails top-realm `instanceof Range`.
|
||||
if (isRangeLike(anchor)) return anchor;
|
||||
if (anchor) {
|
||||
const range = doc.createRange();
|
||||
range.selectNodeContents(anchor);
|
||||
@@ -182,6 +260,24 @@ export const useParagraphMode = ({ bookKey, viewRef }: UseParagraphModeProps) =>
|
||||
}
|
||||
|
||||
updateStateFromIterator(false);
|
||||
|
||||
// Apply a pending cross-section TTS sync once the iterator targets the
|
||||
// CFI's section and that sync is still the latest sequence seen.
|
||||
const pending = pendingSyncRef.current;
|
||||
if (
|
||||
pending &&
|
||||
followingTtsRef.current &&
|
||||
pending.sectionIndex === currentDocIndexRef.current &&
|
||||
pending.sequence >= lastSequenceSeenRef.current
|
||||
) {
|
||||
pendingSyncRef.current = null;
|
||||
const action = decideParagraphTtsHighlight({
|
||||
kind: pending.kind,
|
||||
hasWordPositions: hasWordPositionsRef.current,
|
||||
});
|
||||
applySyncCfiRef.current?.(pending.cfi, action !== 'skip');
|
||||
refreshTtsSyncStatusRef.current?.();
|
||||
}
|
||||
return true;
|
||||
} finally {
|
||||
isProcessingRef.current = false;
|
||||
@@ -236,6 +332,113 @@ export const useParagraphMode = ({ bookKey, viewRef }: UseParagraphModeProps) =>
|
||||
});
|
||||
}, [getViewSettings, viewRef]);
|
||||
|
||||
// Sync-focus path for TTS following. Moves focus to `index` and scrolls/emits
|
||||
// exactly like focusCurrentParagraph but WITHOUT arming isFocusingRef. The
|
||||
// 200ms isFocusingRef window makes the relocate handler skip re-init; if a
|
||||
// TTS-driven focus armed it, the subsequent TTS section-change relocate would
|
||||
// be eaten and the iterator would never re-init for the new section (it would
|
||||
// then focus paragraph 0 of the wrong section). Re-entrancy from this
|
||||
// programmatic scroll's own relocate is instead prevented by the
|
||||
// lastSequenceSeen / section-match guards on the tts-position handler.
|
||||
const focusParagraphForSync = useCallback(
|
||||
async (index: number) => {
|
||||
const view = viewRef.current;
|
||||
const iterator = iteratorRef.current;
|
||||
if (!view || !iterator) return;
|
||||
|
||||
const range = iterator.goTo(index);
|
||||
if (!range) return;
|
||||
updateStateFromIterator();
|
||||
|
||||
await new Promise((r) => requestAnimationFrame(r));
|
||||
|
||||
const presentation = getParagraphPresentation(
|
||||
range.startContainer.ownerDocument,
|
||||
range,
|
||||
getViewSettings(bookKeyRef.current),
|
||||
);
|
||||
|
||||
const docIndex = currentDocIndexRef.current;
|
||||
const renderer = view.renderer as FoliateView['renderer'] & {
|
||||
goTo?: (target: { index: number; anchor: Range }) => Promise<void>;
|
||||
};
|
||||
if (docIndex !== undefined && renderer.goTo) {
|
||||
renderer.goTo({ index: docIndex, anchor: range });
|
||||
} else {
|
||||
view.renderer.scrollToAnchor?.(range);
|
||||
}
|
||||
|
||||
eventDispatcher.dispatch('paragraph-focus', {
|
||||
bookKey: bookKeyRef.current,
|
||||
range,
|
||||
index: iterator.currentIndex,
|
||||
total: iterator.length,
|
||||
presentation,
|
||||
});
|
||||
},
|
||||
[getViewSettings, viewRef, updateStateFromIterator],
|
||||
);
|
||||
|
||||
// Resolve a TTS cfi to the matching paragraph index in the current section and
|
||||
// sync-focus it. No-op when the cfi can't be resolved or maps nowhere (-1).
|
||||
// When `highlight` is set, also dispatch the spoken word/sentence offsets so
|
||||
// the overlay highlights the current text within the focused paragraph (#3235);
|
||||
// this fires even when the paragraph doesn't change (word moving within it).
|
||||
const applySyncCfi = useCallback(
|
||||
(cfi: string, highlight: boolean) => {
|
||||
const view = viewRef.current;
|
||||
const iterator = iteratorRef.current;
|
||||
const docIndex = currentDocIndexRef.current;
|
||||
if (!view || !iterator || docIndex === undefined) return;
|
||||
|
||||
let range: Range | null = null;
|
||||
try {
|
||||
const resolved = view.resolveCFI(cfi);
|
||||
if (!resolved || resolved.index !== docIndex) return;
|
||||
const content = getPrimaryContent();
|
||||
const doc = content?.doc;
|
||||
if (!doc) return;
|
||||
const anchor = resolved.anchor(doc);
|
||||
// Realm-safe: iframe-realm Range fails top-realm `instanceof Range`.
|
||||
if (isRangeLike(anchor)) {
|
||||
range = anchor;
|
||||
} else if (anchor) {
|
||||
range = doc.createRange();
|
||||
range.selectNodeContents(anchor);
|
||||
}
|
||||
} catch {
|
||||
return;
|
||||
}
|
||||
if (!range) return;
|
||||
|
||||
const index = iterator.findIndexByRange(range, iterator.currentIndex);
|
||||
if (index < 0) return;
|
||||
|
||||
// Move focus only when the spoken position crossed into another paragraph;
|
||||
// goTo() runs synchronously so iterator.current() is the target afterwards.
|
||||
if (index !== iterator.currentIndex) {
|
||||
focusParagraphForSync(index);
|
||||
}
|
||||
|
||||
if (highlight) {
|
||||
const paragraphRange = iterator.current();
|
||||
const offsets = paragraphRange
|
||||
? computeParagraphHighlightOffsets(paragraphRange, range)
|
||||
: null;
|
||||
if (offsets) {
|
||||
eventDispatcher.dispatch('paragraph-tts-highlight', {
|
||||
bookKey: bookKeyRef.current,
|
||||
index,
|
||||
start: offsets.start,
|
||||
end: offsets.end,
|
||||
});
|
||||
}
|
||||
}
|
||||
},
|
||||
[viewRef, getPrimaryContent, focusParagraphForSync],
|
||||
);
|
||||
applySyncCfiRef.current = applySyncCfi;
|
||||
|
||||
const waitForNewSection = useCallback(
|
||||
async (oldIndex: number | undefined, maxAttempts: number = 15): Promise<boolean> => {
|
||||
const view = viewRef.current;
|
||||
@@ -262,6 +465,11 @@ export const useParagraphMode = ({ bookKey, viewRef }: UseParagraphModeProps) =>
|
||||
const view = viewRef.current;
|
||||
if (!iterator || !view) return false;
|
||||
|
||||
// Manual nav decouples from TTS following until the user re-engages.
|
||||
followingTtsRef.current = false;
|
||||
pendingSyncRef.current = null;
|
||||
refreshTtsSyncStatus();
|
||||
|
||||
const range = iterator.next();
|
||||
if (range) {
|
||||
updateStateFromIterator();
|
||||
@@ -302,13 +510,25 @@ export const useParagraphMode = ({ bookKey, viewRef }: UseParagraphModeProps) =>
|
||||
focusCurrentParagraph();
|
||||
return false;
|
||||
}
|
||||
}, [viewRef, updateStateFromIterator, focusCurrentParagraph, initIterator, waitForNewSection]);
|
||||
}, [
|
||||
viewRef,
|
||||
updateStateFromIterator,
|
||||
focusCurrentParagraph,
|
||||
initIterator,
|
||||
waitForNewSection,
|
||||
refreshTtsSyncStatus,
|
||||
]);
|
||||
|
||||
const goToPrevParagraph = useCallback(async () => {
|
||||
const iterator = iteratorRef.current;
|
||||
const view = viewRef.current;
|
||||
if (!iterator || !view) return false;
|
||||
|
||||
// Manual nav decouples from TTS following until the user re-engages.
|
||||
followingTtsRef.current = false;
|
||||
pendingSyncRef.current = null;
|
||||
refreshTtsSyncStatus();
|
||||
|
||||
const range = iterator.prev();
|
||||
if (range) {
|
||||
updateStateFromIterator();
|
||||
@@ -349,7 +569,43 @@ export const useParagraphMode = ({ bookKey, viewRef }: UseParagraphModeProps) =>
|
||||
focusCurrentParagraph();
|
||||
return false;
|
||||
}
|
||||
}, [viewRef, updateStateFromIterator, focusCurrentParagraph, initIterator, waitForNewSection]);
|
||||
}, [
|
||||
viewRef,
|
||||
updateStateFromIterator,
|
||||
focusCurrentParagraph,
|
||||
initIterator,
|
||||
waitForNewSection,
|
||||
refreshTtsSyncStatus,
|
||||
]);
|
||||
|
||||
// Re-engage TTS following after a manual nav decoupled it (indicator's
|
||||
// "Resume audio" action). Sets following back on and refreshes the derived
|
||||
// status; the next tts-position event re-syncs the focused paragraph. No-op
|
||||
// on fixed-layout (sync is unsupported there).
|
||||
const reengageTtsFollow = useCallback(() => {
|
||||
if (isFixedLayout) return;
|
||||
followingTtsRef.current = true;
|
||||
refreshTtsSyncStatus();
|
||||
}, [isFixedLayout, refreshTtsSyncStatus]);
|
||||
|
||||
// Audio (TTS) toggle from the paragraph bar (#3235). When a TTS session is
|
||||
// engaged, stop it; otherwise start it from the FOCUSED paragraph with
|
||||
// start-alignment — the paragraph's range (validated live) + its section index
|
||||
// — so audio begins at the same paragraph that's highlighted. Mirrors RSVP's
|
||||
// handleToggleTtsAudio.
|
||||
const toggleTtsAudio = useCallback(() => {
|
||||
if (ttsActiveRef.current) {
|
||||
eventDispatcher.dispatch('tts-stop', { bookKey: bookKeyRef.current });
|
||||
return;
|
||||
}
|
||||
const range = iteratorRef.current?.current() ?? null;
|
||||
const docIndex = currentDocIndexRef.current;
|
||||
// The doc the focused paragraph lives in (current primary content), used to
|
||||
// validate the range is live before passing it along.
|
||||
const currentDoc = getPrimaryContent()?.doc;
|
||||
const detail = buildParagraphTtsSpeakDetail(range, docIndex, bookKeyRef.current, currentDoc);
|
||||
eventDispatcher.dispatch('tts-speak', detail);
|
||||
}, [getPrimaryContent]);
|
||||
|
||||
const goToParagraph = useCallback(
|
||||
(index: number) => {
|
||||
@@ -509,6 +765,93 @@ export const useParagraphMode = ({ bookKey, viewRef }: UseParagraphModeProps) =>
|
||||
};
|
||||
}, [toggleParagraphMode, goToNextParagraph, goToPrevParagraph, paragraphConfig.enabled]);
|
||||
|
||||
// TTS sync (one-way follower): while paragraph mode is active, follow the
|
||||
// spoken position. TTS is the clock; this never drives TTS back. Fixed-layout
|
||||
// books never engage (D7); the indicator stays 'unsupported'.
|
||||
useEffect(() => {
|
||||
if (!paragraphConfig.enabled) return;
|
||||
if (isFixedLayout) return;
|
||||
|
||||
const handlePlaybackState = (event: CustomEvent) => {
|
||||
const detail = event.detail as { bookKey?: string; state?: string } | undefined;
|
||||
if (detail?.bookKey !== bookKeyRef.current) return;
|
||||
const playing = detail.state === 'playing';
|
||||
ttsPlayingRef.current = playing;
|
||||
// A session exists while playing OR paused; only a full stop clears it.
|
||||
const active = detail.state === 'playing' || detail.state === 'paused';
|
||||
ttsActiveRef.current = active;
|
||||
setTtsActive(active);
|
||||
if (playing) {
|
||||
// Fresh engage (re-)enables following.
|
||||
followingTtsRef.current = true;
|
||||
}
|
||||
if (!active) {
|
||||
// Full stop: forget word-boundary state and clear the word highlight so a
|
||||
// later engine (which may lack word boundaries) starts clean.
|
||||
hasWordPositionsRef.current = false;
|
||||
eventDispatcher.dispatch('paragraph-tts-highlight', {
|
||||
bookKey: bookKeyRef.current,
|
||||
clear: true,
|
||||
});
|
||||
}
|
||||
refreshTtsSyncStatus();
|
||||
};
|
||||
|
||||
const handlePosition = (event: CustomEvent) => {
|
||||
const detail = event.detail as
|
||||
| {
|
||||
bookKey?: string;
|
||||
cfi?: string;
|
||||
sectionIndex?: number;
|
||||
sequence?: number;
|
||||
kind?: 'word' | 'sentence';
|
||||
}
|
||||
| undefined;
|
||||
if (detail?.bookKey !== bookKeyRef.current) return;
|
||||
if (!followingTtsRef.current) return;
|
||||
if (typeof detail.cfi !== 'string' || typeof detail.sectionIndex !== 'number') return;
|
||||
|
||||
// Drop out-of-order / stale events (dispatch awaits listeners serially and
|
||||
// callers fire-and-forget, so a slow map can land after a newer one).
|
||||
const sequence = detail.sequence ?? -Infinity;
|
||||
if (sequence <= lastSequenceSeenRef.current) return;
|
||||
lastSequenceSeenRef.current = sequence;
|
||||
|
||||
// Word vs sentence highlight granularity (Edge emits both; words win once
|
||||
// seen). Paragraph selection still runs for 'skip' so following keeps up.
|
||||
const action = decideParagraphTtsHighlight({
|
||||
kind: detail.kind,
|
||||
hasWordPositions: hasWordPositionsRef.current,
|
||||
});
|
||||
if (detail.kind === 'word') hasWordPositionsRef.current = true;
|
||||
|
||||
if (detail.sectionIndex === currentDocIndexRef.current) {
|
||||
applySyncCfi(detail.cfi, action !== 'skip');
|
||||
return;
|
||||
}
|
||||
|
||||
// Different section: don't map. Stash the latest sync and invalidate the
|
||||
// current section so the existing relocate handler re-inits the iterator
|
||||
// for the new section; the pending sync is applied once re-init completes.
|
||||
pendingSyncRef.current = {
|
||||
cfi: detail.cfi,
|
||||
sequence,
|
||||
sectionIndex: detail.sectionIndex,
|
||||
kind: detail.kind,
|
||||
};
|
||||
iteratorRef.current = null;
|
||||
refreshTtsSyncStatus();
|
||||
};
|
||||
|
||||
eventDispatcher.on('tts-playback-state', handlePlaybackState);
|
||||
eventDispatcher.on('tts-position', handlePosition);
|
||||
|
||||
return () => {
|
||||
eventDispatcher.off('tts-playback-state', handlePlaybackState);
|
||||
eventDispatcher.off('tts-position', handlePosition);
|
||||
};
|
||||
}, [paragraphConfig.enabled, isFixedLayout, applySyncCfi, refreshTtsSyncStatus]);
|
||||
|
||||
useEffect(() => {
|
||||
return () => {
|
||||
if (focusResetTimerRef.current) {
|
||||
@@ -524,11 +867,15 @@ export const useParagraphMode = ({ bookKey, viewRef }: UseParagraphModeProps) =>
|
||||
|
||||
return {
|
||||
paragraphState,
|
||||
ttsSyncStatus,
|
||||
ttsActive,
|
||||
paragraphConfig,
|
||||
toggleParagraphMode,
|
||||
goToNextParagraph,
|
||||
goToPrevParagraph,
|
||||
goToParagraph,
|
||||
toggleTtsAudio,
|
||||
reengageTtsFollow,
|
||||
focusCurrentParagraph,
|
||||
initIterator,
|
||||
};
|
||||
|
||||
@@ -66,6 +66,12 @@ export const useTTSControl = ({ bookKey, onRequestHidePanel }: UseTTSControlProp
|
||||
deinitMediaSession,
|
||||
} = useTTSMediaSession({ bookKey });
|
||||
|
||||
// Broadcast playback transitions on the app-wide bus so consumers that
|
||||
// can't read the hook-local isPlaying flag (RSVP, paragraph mode) can react.
|
||||
const emitPlaybackState = (state: 'playing' | 'paused' | 'stopped') => {
|
||||
eventDispatcher.dispatch('tts-playback-state', { bookKey, state });
|
||||
};
|
||||
|
||||
const handleTTSForward = async (event: CustomEvent) => {
|
||||
const detail = event.detail as { bookKey: string; byMark?: boolean } | undefined;
|
||||
if (detail?.bookKey !== bookKey) return;
|
||||
@@ -92,6 +98,22 @@ export const useTTSControl = ({ bookKey, onRequestHidePanel }: UseTTSControlProp
|
||||
eventDispatcher.dispatch('create-tts-highlight', { bookKey, ...sentence });
|
||||
};
|
||||
|
||||
// Set the TTS rate from the app bus. The RSVP overlay is full-screen, so its
|
||||
// rate picker can't reach the TTS panel; it dispatches `tts-set-rate` and we
|
||||
// reuse the same controller rate-change path the panel uses (handleSetRate,
|
||||
// defined below — stop→setRate→start while playing, throttled). Also persists
|
||||
// the value to viewSettings so it survives like a panel change.
|
||||
const handleTTSSetRate = (event: CustomEvent) => {
|
||||
const detail = event.detail as { bookKey: string; rate?: number } | undefined;
|
||||
if (detail?.bookKey !== bookKey || typeof detail.rate !== 'number') return;
|
||||
const viewSettings = getViewSettings(bookKey);
|
||||
if (viewSettings) {
|
||||
viewSettings.ttsRate = detail.rate;
|
||||
setViewSettings(bookKey, viewSettings);
|
||||
}
|
||||
handleSetRate(detail.rate);
|
||||
};
|
||||
|
||||
const handleTTSTogglePlay = async (event: CustomEvent) => {
|
||||
const detail = event.detail as { bookKey: string } | undefined;
|
||||
if (detail?.bookKey !== bookKey) return;
|
||||
@@ -100,10 +122,12 @@ export const useTTSControl = ({ bookKey, onRequestHidePanel }: UseTTSControlProp
|
||||
if (ttsController.state === 'playing') {
|
||||
setIsPlaying(false);
|
||||
setIsPaused(true);
|
||||
emitPlaybackState('paused');
|
||||
await ttsController.pause();
|
||||
} else {
|
||||
setIsPlaying(true);
|
||||
setIsPaused(false);
|
||||
emitPlaybackState('playing');
|
||||
if (ttsController.state === 'paused') {
|
||||
await ttsController.resume();
|
||||
} else {
|
||||
@@ -118,6 +142,7 @@ export const useTTSControl = ({ bookKey, onRequestHidePanel }: UseTTSControlProp
|
||||
eventDispatcher.on('tts-forward', handleTTSForward);
|
||||
eventDispatcher.on('tts-backward', handleTTSBackward);
|
||||
eventDispatcher.on('tts-toggle-play', handleTTSTogglePlay);
|
||||
eventDispatcher.on('tts-set-rate', handleTTSSetRate);
|
||||
eventDispatcher.on('tts-highlight-sentence', handleTTSHighlightSentence);
|
||||
return () => {
|
||||
eventDispatcher.off('tts-speak', handleTTSSpeak);
|
||||
@@ -125,6 +150,7 @@ export const useTTSControl = ({ bookKey, onRequestHidePanel }: UseTTSControlProp
|
||||
eventDispatcher.off('tts-forward', handleTTSForward);
|
||||
eventDispatcher.off('tts-backward', handleTTSBackward);
|
||||
eventDispatcher.off('tts-toggle-play', handleTTSTogglePlay);
|
||||
eventDispatcher.off('tts-set-rate', handleTTSSetRate);
|
||||
eventDispatcher.off('tts-highlight-sentence', handleTTSHighlightSentence);
|
||||
if (ttsControllerRef.current) {
|
||||
ttsControllerRef.current.shutdown();
|
||||
@@ -293,15 +319,31 @@ export const useTTSControl = ({ bookKey, onRequestHidePanel }: UseTTSControlProp
|
||||
}
|
||||
};
|
||||
|
||||
// Republish the controller's canonical position signal onto the app-wide
|
||||
// bus so paragraph mode + RSVP can follow TTS without touching the
|
||||
// controller. This MUST be its own listener: handleHighlightMark /
|
||||
// handleHighlightWord early-return on following-suppression and text
|
||||
// selection, which would silently stop the modes from following. The
|
||||
// forward fires on every controller 'tts-position', gated only by the
|
||||
// listener's lifecycle (it exists only while the controller does).
|
||||
const handlePosition = (e: Event) => {
|
||||
eventDispatcher.dispatch('tts-position', {
|
||||
bookKey,
|
||||
...(e as CustomEvent).detail,
|
||||
});
|
||||
};
|
||||
|
||||
ttsController.addEventListener('tts-need-auth', handleNeedAuth);
|
||||
ttsController.addEventListener('tts-speak-mark', handleSpeakMark);
|
||||
ttsController.addEventListener('tts-highlight-mark', handleHighlightMark);
|
||||
ttsController.addEventListener('tts-highlight-word', handleHighlightWord);
|
||||
ttsController.addEventListener('tts-position', handlePosition);
|
||||
return () => {
|
||||
ttsController.removeEventListener('tts-need-auth', handleNeedAuth);
|
||||
ttsController.removeEventListener('tts-speak-mark', handleSpeakMark);
|
||||
ttsController.removeEventListener('tts-highlight-mark', handleHighlightMark);
|
||||
ttsController.removeEventListener('tts-highlight-word', handleHighlightWord);
|
||||
ttsController.removeEventListener('tts-position', handlePosition);
|
||||
};
|
||||
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||
}, [ttsController, bookKey]);
|
||||
@@ -473,6 +515,7 @@ export const useTTSControl = ({ bookKey, onRequestHidePanel }: UseTTSControlProp
|
||||
setTtsController(null);
|
||||
getView(bookKey)?.deselect();
|
||||
setIsPlaying(false);
|
||||
emitPlaybackState('stopped');
|
||||
onRequestHidePanel?.();
|
||||
setShowIndicator(false);
|
||||
setShowBackToCurrentTTSLocation(false);
|
||||
@@ -583,6 +626,7 @@ export const useTTSControl = ({ bookKey, onRequestHidePanel }: UseTTSControlProp
|
||||
if (ssml) {
|
||||
const lang = parseSSMLLang(ssml, primaryLang) || 'en';
|
||||
setIsPlaying(true);
|
||||
emitPlaybackState('playing');
|
||||
setTtsLang(lang);
|
||||
|
||||
ttsController.setLang(lang);
|
||||
@@ -619,10 +663,12 @@ export const useTTSControl = ({ bookKey, onRequestHidePanel }: UseTTSControlProp
|
||||
if (isPlaying) {
|
||||
setIsPlaying(false);
|
||||
setIsPaused(true);
|
||||
emitPlaybackState('paused');
|
||||
await ttsController.pause();
|
||||
} else if (isPaused) {
|
||||
setIsPlaying(true);
|
||||
setIsPaused(false);
|
||||
emitPlaybackState('playing');
|
||||
// start for forward/backward/setvoice-paused
|
||||
// set rate don't pause the tts
|
||||
if (ttsController.state === 'paused') {
|
||||
@@ -661,8 +707,10 @@ export const useTTSControl = ({ bookKey, onRequestHidePanel }: UseTTSControlProp
|
||||
if (ttsController) {
|
||||
setIsPlaying(false);
|
||||
setIsPaused(true);
|
||||
emitPlaybackState('paused');
|
||||
await ttsController.pause();
|
||||
}
|
||||
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||
}, []);
|
||||
|
||||
// Rate/voice/timeout/bar controls
|
||||
|
||||
Reference in New Issue
Block a user