forked from akai/readest
* docs(reader): TTS-sync design spec for paragraph mode + RSVP (#3235) Hardened via brainstorming + /autoplan (CEO/Design/Eng dual-voice review). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(tts): emit canonical tts-position event from TTSController (#3235) Controller emits { cfi, kind, sectionIndex, sequence } alongside the existing tts-highlight-mark/-word events. Monotonic sequence lets downstream consumers (paragraph mode, RSVP — later slices) drop out-of-order positions. Additive; existing events untouched. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(reader): containment+cursor CFI->index mappers for TTS sync (#3235) RSVPController.syncToCfi + setExternallyDriven: containment match (fixes mid-token skip), monotonic cursor + binary search (avoids O(N)-per-word jank, no per-word getCFI), -1/no-op on no match (no silent jump to word 0), timer suspension while externally driven. ParagraphIterator.findIndexByRange: hinted + binary-search containment mapper returning -1 on no match (never first()). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(tts): forward tts-position + tts-playback-state onto the app bus (#3235) useTTSControl republishes the controller's canonical tts-position (tagged with bookKey) via a dedicated listener — NOT inside the suppression-gated highlight handlers, so page-follow suppression can't silently desync the modes. Adds tts-playback-state (playing/paused/stopped) so RSVP can track playback without the hook-local isPlaying. Verified by extending the real-foliate-view browser harness. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(reader): paragraph mode follows TTS playback (#3235) When paragraph mode + TTS are both active, the focused paragraph follows the spoken position (sentence granularity, all engines). Section-generation contract (stash cross-section position, apply after the iterator re-inits); sync-focus path that does NOT arm isFocusingRef (avoids the relocate-eaten wrong-section paragraph-0 bug); stale-sequence drop; decouple on manual nav, re-engage on next playing. Start-alignment + visible indicator deferred to later slices. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(rsvp): speed reader follows TTS playback (#3235) Edge word-boundary voices: RSVP shows the spoken word via syncToCfi. Non-Edge (sentence-only) voices: sentence-paced estimator (clamp 60..600 wpm from voice rate, hold at +60 words cap, snap to first word on each new sentence mark). RSVP auto-advance suspended while TTS-driven. Decouple on manual nav via a rsvp-manual-nav signal; re-engage on next playing. Cross-section positions re-extract then apply. Pure decideRsvpTtsPosition helper unit-tested. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(reader): fixed-layout gate + ttsSyncStatus for TTS sync (#3235) Gate sync to reflowable books (D7): fixed-layout reports 'unsupported' and never engages. Both modes expose ttsSyncStatus (idle/following/syncing/ decoupled/unsupported) as the data source for the upcoming indicator. RSVPControl now forwardRef-exposes the status via an imperative handle. Cross-bookKey events ignored (regression-tested). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(reader): 'following audio' indicator for TTS sync (#3235) 5-state pill (following/syncing/decoupled, idle+unsupported render null) shown top-center in the paragraph overlay and as a status row in the RSVP overlay. Decoupled state is the tap-to-resume control; first decouple fires a one-time toast. eink-bordered, glyph+text (no color-only), RTL logical props, touch targets, safe-area top inset. RSVP 'plain' variant matches its themed surface; non-Edge shows '· estimated'. New i18n keys need extraction before merge. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(rsvp): in-overlay TTS toggle + audio-paced speed control (#3235) Voice-glyph audio toggle in the RSVP control row (trailing, by the gear) starts/ stops read-along from inside the full-screen overlay, start-aligned to the current word (range validated against the live doc). While TTS-driven, the WPM control shows a locked 'Audio pace' affordance that opens a compact rate picker; rate changes go through a new tts-set-rate bus event reusing the existing throttled setRate path. Pure buildRsvpTtsSpeakDetail helper unit-tested. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(reader): e2e paragraph mode follows TTS across a section boundary (#3235) Real <foliate-view> browser e2e: with paragraph mode active, the focused paragraph follows the TTS walk and re-targets to the new section after a Ch4->Ch5 boundary (proves no stuck wrong-section paragraph-0 / isFocusingRef trap). Asserts on the owning section of the current range. Test-only. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * i18n(reader): translate TTS-sync strings across 33 locales (#3235) Following audio / · estimated / Resume audio / Stopped following audio / Play audio / Pause audio / Audio pace / Speed follows audio. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(reader): resolve TTS CFI anchors across iframe realms (#3235) RSVP and paragraph follow silently failed to track the spoken word: the CFI anchor from view.resolveCFI(...).anchor(doc) is a Range created in the book iframe's realm, so 'anchor instanceof Range' (top realm) was always false (cross-realm instanceof) -> resolveCfiToRange/applySyncCfi returned null -> syncToCfi never advanced. Add isRangeLike() duck-type (cloneRange is unique to Range) and use it at all 4 CFI-resolution sites. Confirmed live via CDP: before = syncToCfi false (frozen); after = exact word map + RSVP follows Edge TTS at ~171 wpm (audio pace). Unit tests reproduce the cross-realm anchor (jsdom is single-realm so the old code passed there but died in the app). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(rsvp): stop estimator/word fight + map transport to TTS play/pause (#3235) Two read-along refinements (verified live via CDP with Edge TTS): 1. No more jump-ahead-then-snap-back flashing. Word-boundary engines (Edge) emit BOTH sentence marks and word boundaries; RSVP was routing sentence -> the estimator (self-paces ~190xrate, up to +60 words ahead) while word positions snapped it back. Now once a word position is seen, sentence positions are ignored and any running estimator is stopped, so words alone drive RSVP. 2. The RSVP transport (center play/pause, Space, center-tap) maps to TTS play/pause while read-along is engaged (tts-toggle-play), instead of RSVP's own suspended timer. Pausing TTS keeps RSVP suspended (no runaway); a full stop releases it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(rsvp): keep indicator on pause + reach dict management from RSVP (#3235) - Pausing read-along no longer dismisses the 'following audio' indicator / 'Audio pace' lock (layout shift). New 'paused' sync status keeps the indicator row and WPM lock present while TTS is engaged-but-paused; only a full stop clears them. Verified live via CDP: pause keeps the layout, no shift. - Dict management is reachable from RSVP: the settings dialog is z-50, far below the full-screen RSVP overlay (z-[10000]), so it opened invisibly behind it. handleManageDictionary now exits RSVP first (position saved/resumable) so management shows over the reader. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(rsvp): show dict management over RSVP instead of exiting it (#3235) Per feedback: opening dictionary management from the RSVP lookup popup no longer closes the speed reader. The settings dialog is raised above the full-screen RSVP overlay (z-[10000] -> SettingsDialog !z-[10050]) so it shows on top, and RSVP's capture-phase keyboard handler bails while the settings dialog is open so its inputs accept Space and Escape closes settings (not RSVP). Verified live via CDP: management opens over RSVP, RSVP stays active behind it, Escape returns to it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(dict): only apply drag-handle margin compensation when the handle shows (#3235) The dictionary sheet header used -mt-4 to compensate for Dialog's drag handle, but that handle is sm:hidden (shown only below sm). On sm+ the handle is display:none, so -mt-4 pulled the header up into the top edge (broken layout when the lookup renders as a sheet on a short/wide window). Mirror the handle's breakpoint: -mt-4 sm:mt-0. Verified live via CDP. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(reader): in-mode TTS audio toggle for paragraph mode (#3235) Paragraph mode already follows TTS, but there was no way to start read-along from inside it. Add an audio toggle to the ParagraphBar (mirroring RSVP's): it starts TTS start-aligned to the focused paragraph (range validated live, + section index) and stops it. Track session-active vs playing so a pause keeps the indicator ('paused' status) instead of collapsing to idle. Pure buildParagraphTtsSpeakDetail helper unit-tested. Verified live via CDP: tapping the icon starts audio from the focused paragraph and the focus follows speech. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(reader): highlight current TTS word/sentence in paragraph mode (#3235) Paragraph mode follows TTS by advancing the focused paragraph, but the spoken word wasn't highlighted within it like normal mode. The overlay renders a CLONE of the paragraph, so the iframe's TTS highlight isn't visible there — reproduce it on the clone with the CSS Custom Highlight API (no DOM mutation, spans inline boundaries natively, leaves the fade-in animation untouched). - TTSController already tags tts-position with kind word|sentence. The hook decides granularity: word boundaries (Edge) drive a per-word highlight; once seen, the coarse sentence event is skipped so the whole sentence doesn't flicker over the current word. Engines without word boundaries (WebSpeech/Native) fall back to the sentence highlight. - Offsets are computed relative to the paragraph start (so they map 1:1 onto the clone's text) and tagged with the paragraph index so a stale highlight never paints the wrong paragraph. Cleared on stop / section change / disabled. - The ::highlight() style mirrors the user's ttsHighlightOptions color+style. Pure helpers (offset math, word/sentence decision, css builder) unit-tested. Verified live via CDP: word highlight tracks Edge word-by-word and follows across paragraph boundaries (news -> ... -> ladies), matching the TTS color. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Huang Xin
@@ -25,7 +25,10 @@ const DictionarySheet: React.FC<DictionarySheetProps> = ({ word, lang, onDismiss
|
||||
dismissible
|
||||
header={
|
||||
<DictionaryResultsHeader
|
||||
headerClassName='-mt-4'
|
||||
// The -mt-4 compensates for Dialog's drag handle, which is `sm:hidden`
|
||||
// (shown only below sm). Mirror that breakpoint so on sm+ (no handle)
|
||||
// the header isn't pulled up into the top edge.
|
||||
headerClassName='-mt-4 sm:mt-0'
|
||||
currentWord={state.currentWord}
|
||||
canGoBack={state.canGoBack}
|
||||
goBack={state.goBack}
|
||||
|
||||
@@ -9,6 +9,7 @@ import {
|
||||
MdKeyboardArrowDown,
|
||||
MdKeyboardArrowUp,
|
||||
} from 'react-icons/md';
|
||||
import { IoVolumeHigh, IoVolumeMediumOutline } from 'react-icons/io5';
|
||||
import { ViewSettings } from '@/types/book';
|
||||
import { Insets } from '@/types/misc';
|
||||
import { useEnv } from '@/context/EnvContext';
|
||||
@@ -30,6 +31,10 @@ interface ParagraphBarProps {
|
||||
onPrev: () => void;
|
||||
onNext: () => void;
|
||||
onClose: () => void;
|
||||
/** True when a TTS session is engaged (playing/paused) — drives the audio glyph. */
|
||||
ttsActive?: boolean;
|
||||
/** Toggle read-along: start TTS from the focused paragraph, or stop it. */
|
||||
onToggleTtsAudio?: () => void;
|
||||
viewSettings?: ViewSettings;
|
||||
gridInsets: Insets;
|
||||
}
|
||||
@@ -51,6 +56,8 @@ const ParagraphBar: React.FC<ParagraphBarProps> = ({
|
||||
onPrev,
|
||||
onNext,
|
||||
onClose,
|
||||
ttsActive = false,
|
||||
onToggleTtsAudio,
|
||||
viewSettings,
|
||||
gridInsets,
|
||||
}) => {
|
||||
@@ -286,6 +293,37 @@ const ParagraphBar: React.FC<ParagraphBarProps> = ({
|
||||
<NextIcon size={iconSize} />
|
||||
</button>
|
||||
|
||||
{onToggleTtsAudio && (
|
||||
<>
|
||||
<div className='bg-base-content/10 mx-1 h-4 w-px' />
|
||||
|
||||
{/* Audio (TTS) toggle — starts read-along from the focused
|
||||
paragraph, or stops it when engaged (#3235). Active state uses a
|
||||
filled glyph + eink-bordered surface so it reads in e-ink
|
||||
without relying on color. */}
|
||||
<button
|
||||
onClick={onToggleTtsAudio}
|
||||
disabled={isLoading}
|
||||
className={clsx(
|
||||
'flex items-center justify-center rounded-full p-1.5',
|
||||
'transition-all duration-200 ease-out active:scale-90',
|
||||
ttsActive
|
||||
? 'text-primary eink-bordered not-eink:bg-base-200'
|
||||
: 'not-eink:hover:bg-base-200',
|
||||
isLoading && 'pointer-events-none opacity-50',
|
||||
)}
|
||||
title={ttsActive ? _('Pause audio') : _('Play audio')}
|
||||
aria-label={ttsActive ? _('Pause audio') : _('Play audio')}
|
||||
>
|
||||
{ttsActive ? (
|
||||
<IoVolumeHigh size={iconSize} aria-hidden='true' />
|
||||
) : (
|
||||
<IoVolumeMediumOutline size={iconSize} aria-hidden='true' />
|
||||
)}
|
||||
</button>
|
||||
</>
|
||||
)}
|
||||
|
||||
<button
|
||||
onClick={onClose}
|
||||
disabled={isLoading}
|
||||
|
||||
@@ -1,9 +1,11 @@
|
||||
'use client';
|
||||
|
||||
import React from 'react';
|
||||
import React, { useEffect, useRef } from 'react';
|
||||
import { FoliateView } from '@/types/view';
|
||||
import { Insets } from '@/types/misc';
|
||||
import { useReaderStore } from '@/store/readerStore';
|
||||
import { useTranslation } from '@/hooks/useTranslation';
|
||||
import { eventDispatcher } from '@/utils/event';
|
||||
import { useParagraphMode } from '../../hooks/useParagraphMode';
|
||||
import ParagraphBar from './ParagraphBar';
|
||||
import ParagraphOverlay from './ParagraphOverlay';
|
||||
@@ -17,17 +19,40 @@ interface ParagraphControlProps {
|
||||
}
|
||||
|
||||
const ParagraphControl: React.FC<ParagraphControlProps> = ({ bookKey, viewRef, gridInsets }) => {
|
||||
const _ = useTranslation();
|
||||
const { getViewSettings } = useReaderStore();
|
||||
const viewSettings = getViewSettings(bookKey);
|
||||
|
||||
const {
|
||||
paragraphState,
|
||||
paragraphConfig,
|
||||
ttsSyncStatus,
|
||||
ttsActive,
|
||||
toggleParagraphMode,
|
||||
goToNextParagraph,
|
||||
goToPrevParagraph,
|
||||
toggleTtsAudio,
|
||||
reengageTtsFollow,
|
||||
} = useParagraphMode({ bookKey, viewRef });
|
||||
|
||||
// One-time-per-session decouple toast: the first time following drops while
|
||||
// TTS still plays, tell the user once. Reset when following re-engages so a
|
||||
// later decouple notifies again.
|
||||
const decoupleToastShownRef = useRef(false);
|
||||
useEffect(() => {
|
||||
if (ttsSyncStatus === 'decoupled') {
|
||||
if (!decoupleToastShownRef.current) {
|
||||
decoupleToastShownRef.current = true;
|
||||
eventDispatcher.dispatch('toast', {
|
||||
message: _('Stopped following audio'),
|
||||
type: 'info',
|
||||
});
|
||||
}
|
||||
} else if (ttsSyncStatus === 'following') {
|
||||
decoupleToastShownRef.current = false;
|
||||
}
|
||||
}, [ttsSyncStatus, _]);
|
||||
|
||||
if (!paragraphConfig?.enabled) {
|
||||
return null;
|
||||
}
|
||||
@@ -39,6 +64,8 @@ const ParagraphControl: React.FC<ParagraphControlProps> = ({ bookKey, viewRef, g
|
||||
dimOpacity={DIM_OPACITY}
|
||||
viewSettings={viewSettings ?? undefined}
|
||||
gridInsets={gridInsets}
|
||||
ttsSyncStatus={ttsSyncStatus}
|
||||
onResumeTtsFollow={reengageTtsFollow}
|
||||
onClose={toggleParagraphMode}
|
||||
/>
|
||||
<ParagraphBar
|
||||
@@ -49,6 +76,8 @@ const ParagraphControl: React.FC<ParagraphControlProps> = ({ bookKey, viewRef, g
|
||||
onPrev={goToPrevParagraph}
|
||||
onNext={goToNextParagraph}
|
||||
onClose={toggleParagraphMode}
|
||||
ttsActive={ttsActive}
|
||||
onToggleTtsAudio={toggleTtsAudio}
|
||||
viewSettings={viewSettings ?? undefined}
|
||||
gridInsets={gridInsets}
|
||||
/>
|
||||
|
||||
@@ -12,12 +12,23 @@ import {
|
||||
getParagraphLayoutContext,
|
||||
ParagraphPresentation,
|
||||
} from '@/utils/paragraphPresentation';
|
||||
import { getTextSubRange } from '@/services/tts/wordHighlight';
|
||||
import TTSFollowIndicator, { TtsSyncStatus } from '../tts/TTSFollowIndicator';
|
||||
import { buildTtsHighlightCssText } from './paragraphTts';
|
||||
|
||||
// CSS Custom Highlight registry name for the in-paragraph TTS word/sentence
|
||||
// highlight (#3235). Unique per app so it never collides with other highlights.
|
||||
const TTS_HIGHLIGHT_NAME = 'readest-tts-paragraph';
|
||||
|
||||
interface ParagraphOverlayProps {
|
||||
bookKey: string;
|
||||
dimOpacity: number;
|
||||
viewSettings?: ViewSettings;
|
||||
gridInsets?: Insets;
|
||||
/** Derived TTS-sync status driving the "following audio" indicator (#3235). */
|
||||
ttsSyncStatus?: TtsSyncStatus;
|
||||
/** Re-engage following after a manual nav decoupled it (indicator action). */
|
||||
onResumeTtsFollow?: () => void;
|
||||
onClose?: () => void;
|
||||
}
|
||||
|
||||
@@ -119,6 +130,8 @@ const ParagraphOverlay: React.FC<ParagraphOverlayProps> = ({
|
||||
dimOpacity,
|
||||
viewSettings,
|
||||
gridInsets = { top: 0, right: 0, bottom: 0, left: 0 },
|
||||
ttsSyncStatus = 'idle',
|
||||
onResumeTtsFollow,
|
||||
onClose,
|
||||
}) => {
|
||||
const { appService } = useEnv();
|
||||
@@ -127,6 +140,14 @@ const ParagraphOverlay: React.FC<ParagraphOverlayProps> = ({
|
||||
const [isOverlayMounted, setIsOverlayMounted] = useState(false);
|
||||
const [isChangingSection, setIsChangingSection] = useState(false);
|
||||
const [sectionDirection, setSectionDirection] = useState<'next' | 'prev'>('next');
|
||||
// Index of the currently focused paragraph, used to gate the TTS word/sentence
|
||||
// highlight so a stale highlight never lands on the wrong paragraph (#3235).
|
||||
const [focusIndex, setFocusIndex] = useState(-1);
|
||||
const [ttsHighlight, setTtsHighlight] = useState<{
|
||||
index: number;
|
||||
start: number;
|
||||
end: number;
|
||||
} | null>(null);
|
||||
const paragraphIdCounter = useRef(0);
|
||||
const containerRef = useRef<HTMLDivElement>(null);
|
||||
const contentRef = useRef<HTMLDivElement>(null);
|
||||
@@ -196,6 +217,12 @@ const ParagraphOverlay: React.FC<ParagraphOverlayProps> = ({
|
||||
}) as React.CSSProperties,
|
||||
[],
|
||||
);
|
||||
// `::highlight()` declaration matching the user's TTS highlight color/style so
|
||||
// the in-paragraph word/sentence highlight looks like normal mode (#3235).
|
||||
const ttsHighlightCss = useMemo(
|
||||
() => buildTtsHighlightCssText(viewSettings?.ttsHighlightOptions),
|
||||
[viewSettings?.ttsHighlightOptions],
|
||||
);
|
||||
const fallbackPresentation = useMemo(
|
||||
(): ParagraphPresentation => ({
|
||||
dir: layoutContext.rtl ? 'rtl' : 'ltr',
|
||||
@@ -245,6 +272,7 @@ const ParagraphOverlay: React.FC<ParagraphOverlayProps> = ({
|
||||
setIsChangingSection(false);
|
||||
setIsVisible(true);
|
||||
setIsOverlayMounted(true);
|
||||
setFocusIndex(typeof event.detail?.index === 'number' ? event.detail.index : -1);
|
||||
addParagraph(range, presentation);
|
||||
}
|
||||
};
|
||||
@@ -257,6 +285,7 @@ const ParagraphOverlay: React.FC<ParagraphOverlayProps> = ({
|
||||
}
|
||||
setIsOverlayMounted(false);
|
||||
setIsChangingSection(false);
|
||||
setTtsHighlight(null);
|
||||
setTimeout(() => {
|
||||
setIsVisible(false);
|
||||
setParagraphs([]);
|
||||
@@ -267,18 +296,36 @@ const ParagraphOverlay: React.FC<ParagraphOverlayProps> = ({
|
||||
if (event.detail?.bookKey !== bookKey) return;
|
||||
setSectionDirection(event.detail?.direction || 'next');
|
||||
setParagraphs([]);
|
||||
setTtsHighlight(null);
|
||||
setIsChangingSection(true);
|
||||
};
|
||||
|
||||
// TTS word/sentence highlight within the focused paragraph (#3235). The hook
|
||||
// sends character offsets relative to the paragraph start (+ its index, to
|
||||
// guard against landing on the wrong paragraph) or a clear when TTS stops.
|
||||
const handleTtsHighlight = (event: CustomEvent) => {
|
||||
if (event.detail?.bookKey !== bookKey) return;
|
||||
const detail = event.detail as
|
||||
| { clear?: boolean; index?: number; start?: number; end?: number }
|
||||
| undefined;
|
||||
if (detail?.clear || typeof detail?.start !== 'number' || typeof detail?.end !== 'number') {
|
||||
setTtsHighlight(null);
|
||||
return;
|
||||
}
|
||||
setTtsHighlight({ index: detail.index ?? -1, start: detail.start, end: detail.end });
|
||||
};
|
||||
|
||||
eventDispatcher.on('paragraph-focus', handleFocus);
|
||||
eventDispatcher.on('paragraph-mode-disabled', handleDisabled);
|
||||
eventDispatcher.on('paragraph-section-changing', handleSectionChanging);
|
||||
eventDispatcher.on('paragraph-tts-highlight', handleTtsHighlight);
|
||||
|
||||
return () => {
|
||||
if (sectionChangeTimeoutId) clearTimeout(sectionChangeTimeoutId);
|
||||
eventDispatcher.off('paragraph-focus', handleFocus);
|
||||
eventDispatcher.off('paragraph-mode-disabled', handleDisabled);
|
||||
eventDispatcher.off('paragraph-section-changing', handleSectionChanging);
|
||||
eventDispatcher.off('paragraph-tts-highlight', handleTtsHighlight);
|
||||
};
|
||||
}, [bookKey, addParagraph]);
|
||||
|
||||
@@ -334,6 +381,39 @@ const ParagraphOverlay: React.FC<ParagraphOverlayProps> = ({
|
||||
return () => window.removeEventListener('wheel', handleWheel, true);
|
||||
}, [isVisible, bookKey]);
|
||||
|
||||
// Paint the current TTS word/sentence onto the cloned paragraph using the CSS
|
||||
// Custom Highlight API (#3235). It highlights a Range without mutating the DOM
|
||||
// and natively spans inline element boundaries (sentences), so the fade-in
|
||||
// animation and the clone's markup stay untouched. Re-runs when the clone
|
||||
// (paragraphs) or the offsets change; gated on the index so a highlight from a
|
||||
// previous paragraph never paints the wrong text. No-op where unsupported.
|
||||
useEffect(() => {
|
||||
const registry = typeof CSS !== 'undefined' ? CSS.highlights : undefined;
|
||||
if (!registry || typeof Highlight === 'undefined') return undefined;
|
||||
const clear = () => {
|
||||
registry.delete(TTS_HIGHLIGHT_NAME);
|
||||
};
|
||||
|
||||
if (!ttsHighlight || ttsHighlight.index !== focusIndex) {
|
||||
clear();
|
||||
return clear;
|
||||
}
|
||||
const contentEl = contentRef.current?.querySelector('.paragraph-content');
|
||||
if (!contentEl) {
|
||||
clear();
|
||||
return clear;
|
||||
}
|
||||
const base = document.createRange();
|
||||
base.selectNodeContents(contentEl);
|
||||
const range = getTextSubRange(base, ttsHighlight.start, ttsHighlight.end);
|
||||
if (!range) {
|
||||
clear();
|
||||
return clear;
|
||||
}
|
||||
registry.set(TTS_HIGHLIGHT_NAME, new Highlight(range));
|
||||
return clear;
|
||||
}, [ttsHighlight, focusIndex, paragraphs]);
|
||||
|
||||
const handleTouchStart = useCallback(
|
||||
(e: React.TouchEvent) => {
|
||||
const touchStartY = e.touches[0]?.clientY ?? 0;
|
||||
@@ -465,6 +545,21 @@ const ParagraphOverlay: React.FC<ParagraphOverlayProps> = ({
|
||||
onTouchStart={handleTouchStart}
|
||||
onKeyDown={(e) => e.stopPropagation()}
|
||||
>
|
||||
{/* TTS "following audio" indicator, pinned top-center. Anchored below the
|
||||
top safe-area inset the overlay already accounts for; idle/unsupported
|
||||
render nothing so it stays out of the way when TTS isn't driving. */}
|
||||
<div
|
||||
className='pointer-events-none absolute inset-x-0 z-10 flex justify-center'
|
||||
style={{
|
||||
top: appService?.hasSafeAreaInset ? `calc(${gridInsets.top}px + 0.75rem)` : '0.75rem',
|
||||
}}
|
||||
>
|
||||
{/* Only the indicator itself takes pointer events (its decoupled state is
|
||||
a button); the wrapper stays transparent to backdrop/region taps. */}
|
||||
<div className='pointer-events-auto'>
|
||||
<TTSFollowIndicator status={ttsSyncStatus} onResume={onResumeTtsFollow} />
|
||||
</div>
|
||||
</div>
|
||||
{/* eslint-disable-next-line jsx-a11y/click-events-have-key-events, jsx-a11y/no-static-element-interactions */}
|
||||
<div
|
||||
ref={contentRef}
|
||||
@@ -492,6 +587,10 @@ const ParagraphOverlay: React.FC<ParagraphOverlayProps> = ({
|
||||
.paragraph-content > :last-child {
|
||||
margin-block-end: 0;
|
||||
}
|
||||
|
||||
::highlight(${TTS_HIGHLIGHT_NAME}) {
|
||||
${ttsHighlightCss}
|
||||
}
|
||||
`}</style>
|
||||
{activeParagraph ? (
|
||||
<div
|
||||
|
||||
@@ -0,0 +1,112 @@
|
||||
import { TTSHighlightOptions } from '@/services/tts/types';
|
||||
|
||||
// Detail payload for the app-bus `tts-speak` event (see useTTSControl.handleTTSSpeak,
|
||||
// which honors a passed `range` + `index`). Paragraph mode starts audio from the
|
||||
// focused paragraph so the listener and the highlighted paragraph stay aligned.
|
||||
// Mirrors rsvpTts.ts (decision 5, #3235).
|
||||
export interface ParagraphTtsSpeakDetail {
|
||||
bookKey: string;
|
||||
// Section spine index of the focused paragraph — starts TTS in the right section.
|
||||
index?: number;
|
||||
// Live DOM range of the focused paragraph — starts TTS at the exact paragraph.
|
||||
// Omitted when there is no range or the range is stale (its document no longer
|
||||
// matches the current content), so TTS falls back to its own start position.
|
||||
range?: Range;
|
||||
}
|
||||
|
||||
// Build the `tts-speak` detail for "start audio from the focused paragraph"
|
||||
// (#3235). Returns `{ bookKey }` only when there is nothing to align to.
|
||||
//
|
||||
// Start-alignment rules (mirror buildRsvpTtsSpeakDetail):
|
||||
// - index = the paragraph's spine index (when known), so audio begins in the
|
||||
// focused section even if the range can't be used.
|
||||
// - range is included ONLY when it is live: it exists and its ownerDocument is
|
||||
// the document paragraph mode is currently rendering (`currentDoc`). A stale
|
||||
// or cross-document range would resolve to the wrong place, so it is dropped
|
||||
// and TTS falls back to its own start position.
|
||||
export const buildParagraphTtsSpeakDetail = (
|
||||
range: Range | null | undefined,
|
||||
docIndex: number | undefined,
|
||||
bookKey: string,
|
||||
currentDoc: Document | null | undefined,
|
||||
): ParagraphTtsSpeakDetail => {
|
||||
const detail: ParagraphTtsSpeakDetail = { bookKey };
|
||||
|
||||
if (typeof docIndex === 'number') {
|
||||
detail.index = docIndex;
|
||||
}
|
||||
|
||||
if (range && currentDoc && range.startContainer.ownerDocument === currentDoc) {
|
||||
detail.range = range;
|
||||
}
|
||||
|
||||
return detail;
|
||||
};
|
||||
|
||||
// Character offsets of a spoken word/sentence range relative to the start of the
|
||||
// focused paragraph's text (#3235). Paragraph mode renders a CLONE of the
|
||||
// paragraph in the overlay, so the iframe's TTS highlight isn't visible there;
|
||||
// the overlay re-creates it from these offsets (which map 1:1 onto the clone's
|
||||
// text content because both the clone and these offsets start at the paragraph
|
||||
// start). Returns null when the target isn't inside the paragraph or is empty.
|
||||
export const computeParagraphHighlightOffsets = (
|
||||
paragraphRange: Range,
|
||||
targetRange: Range,
|
||||
): { start: number; end: number } | null => {
|
||||
try {
|
||||
const doc = paragraphRange.startContainer.ownerDocument;
|
||||
if (!doc) return null;
|
||||
// The target must begin within the paragraph; isPointInRange returns false
|
||||
// (no throw) for points before/after the range or in a different root.
|
||||
if (!paragraphRange.isPointInRange(targetRange.startContainer, targetRange.startOffset)) {
|
||||
return null;
|
||||
}
|
||||
const pre = doc.createRange();
|
||||
pre.setStart(paragraphRange.startContainer, paragraphRange.startOffset);
|
||||
pre.setEnd(targetRange.startContainer, targetRange.startOffset);
|
||||
const start = pre.toString().length;
|
||||
const length = targetRange.toString().length;
|
||||
if (length <= 0) return null;
|
||||
return { start, end: start + length };
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
};
|
||||
|
||||
export type ParagraphTtsHighlightAction = 'word' | 'sentence' | 'skip';
|
||||
|
||||
// Decide what to highlight for a `tts-position` event (#3235). Edge TTS emits
|
||||
// BOTH a per-sentence mark and per-word boundaries; once words have been seen we
|
||||
// keep the fine-grained word highlight and skip the coarse sentence event so the
|
||||
// whole sentence doesn't flicker over the current word. Engines without word
|
||||
// boundaries (WebSpeech/Native) only emit sentence events → sentence highlight.
|
||||
export const decideParagraphTtsHighlight = (input: {
|
||||
kind?: 'word' | 'sentence';
|
||||
hasWordPositions: boolean;
|
||||
}): ParagraphTtsHighlightAction => {
|
||||
if (input.kind === 'word') return 'word';
|
||||
if (input.hasWordPositions) return 'skip';
|
||||
return 'sentence';
|
||||
};
|
||||
|
||||
// CSS declaration body for the overlay's `::highlight()` rule, derived from the
|
||||
// user's TTS highlight options so the in-paragraph highlight matches normal mode
|
||||
// (#3235). The CSS Custom Highlight pseudo only supports a handful of properties
|
||||
// (background-color, text-decoration, color, …), so styles map onto those:
|
||||
// - highlight/outline → a translucent background of the chosen color
|
||||
// - underline/squiggly/strikethrough → a text-decoration in the chosen color
|
||||
export const buildTtsHighlightCssText = (options?: TTSHighlightOptions): string => {
|
||||
const color = options?.color || '#808080';
|
||||
switch (options?.style) {
|
||||
case 'underline':
|
||||
return `text-decoration: underline; text-decoration-color: ${color}; text-decoration-thickness: 2px; text-underline-offset: 2px;`;
|
||||
case 'squiggly':
|
||||
return `text-decoration: underline wavy; text-decoration-color: ${color};`;
|
||||
case 'strikethrough':
|
||||
return `text-decoration: line-through; text-decoration-color: ${color};`;
|
||||
case 'highlight':
|
||||
case 'outline':
|
||||
default:
|
||||
return `background-color: color-mix(in srgb, ${color} 40%, transparent);`;
|
||||
}
|
||||
};
|
||||
@@ -1,6 +1,6 @@
|
||||
'use client';
|
||||
|
||||
import React, { useState, useRef, useEffect, useCallback } from 'react';
|
||||
import { useState, useRef, useEffect, useCallback, useImperativeHandle, forwardRef } from 'react';
|
||||
import { createPortal } from 'react-dom';
|
||||
import { useReaderStore } from '@/store/readerStore';
|
||||
import { useBookProgress } from '@/store/readerProgressStore';
|
||||
@@ -14,6 +14,7 @@ import {
|
||||
buildRsvpExitConfigUpdate,
|
||||
} from '@/services/rsvp';
|
||||
import { eventDispatcher } from '@/utils/event';
|
||||
import { buildRsvpTtsSpeakDetail } from './rsvpTts';
|
||||
import { getBaseFontFamily } from '@/utils/style';
|
||||
import { useEnv } from '@/context/EnvContext';
|
||||
import { useTranslation } from '@/hooks/useTranslation';
|
||||
@@ -29,6 +30,140 @@ interface RSVPControlProps {
|
||||
gridInsets: Insets;
|
||||
}
|
||||
|
||||
// Imperative handle so the later TTS-sync indicator slice (and tests) can read
|
||||
// the derived sync status without RSVPControl having a sibling to surface it to.
|
||||
export interface RSVPControlHandle {
|
||||
ttsSyncStatus: TtsSyncStatus;
|
||||
}
|
||||
|
||||
// ─── TTS-sync decision logic (slice 5, #3235) ──────────────────────────
|
||||
// RSVP follows the spoken position while TTS plays (TTS is the clock). The
|
||||
// pure decision below maps an incoming tts-position event to the action the
|
||||
// component should take, so the stale-sequence / decouple / section-stash
|
||||
// rules can be unit-tested without mounting the heavy component.
|
||||
|
||||
export interface RsvpTtsPositionDetail {
|
||||
bookKey?: string;
|
||||
cfi?: string;
|
||||
kind?: 'word' | 'sentence';
|
||||
sectionIndex?: number;
|
||||
sequence?: number;
|
||||
}
|
||||
|
||||
// Derived sync state surfaced to RSVPOverlay for the later indicator slice.
|
||||
// idle — RSVP not following TTS (not playing / not engaged)
|
||||
// following — engaged and mapping the spoken position
|
||||
// syncing — a cross-section re-extract is pending
|
||||
// decoupled — a manual RSVP nav dropped following while TTS still plays
|
||||
// unsupported — fixed-layout book; TTS sync can never engage (D7)
|
||||
export type TtsSyncStatus =
|
||||
| 'idle'
|
||||
| 'following'
|
||||
| 'syncing'
|
||||
| 'decoupled'
|
||||
| 'paused'
|
||||
| 'unsupported';
|
||||
|
||||
export interface RsvpTtsPendingSync {
|
||||
cfi: string;
|
||||
sequence: number;
|
||||
sectionIndex: number;
|
||||
}
|
||||
|
||||
export interface RsvpTtsSyncState {
|
||||
// Whether RSVP is currently following TTS. A manual RSVP nav decouples
|
||||
// (sets false); the next 'playing' re-engages.
|
||||
following: boolean;
|
||||
// Monotonic guard: drop tts-position with sequence <= this.
|
||||
lastSequenceSeen: number;
|
||||
// The section RSVP currently has extracted words for.
|
||||
currentSectionIndex: number;
|
||||
// Latest sync stashed during a pending section re-extract.
|
||||
pendingSync?: RsvpTtsPendingSync;
|
||||
// True once a word-level position has arrived — i.e. the engine emits word
|
||||
// boundaries (Edge). Word-boundary engines ALSO emit sentence marks, so once
|
||||
// this is set we must NOT drive the estimator on sentence positions (it
|
||||
// outruns the audio and the word positions snap RSVP back → flashing). Reset
|
||||
// on a full stop so a later voice switch (e.g. to a sentence-only engine)
|
||||
// re-enables the estimator.
|
||||
hasWordPositions: boolean;
|
||||
}
|
||||
|
||||
export interface RsvpTtsPositionDecision {
|
||||
// sync → Edge word-level: controller.syncToCfi(cfi)
|
||||
// drive-estimator→ non-Edge sentence: controller.driveEstimatedFromCfi(cfi)
|
||||
// reextract → different section: trigger re-extract, apply stash after
|
||||
// ignore → drop (wrong book / stale seq / decoupled / malformed /
|
||||
// fixed-layout where sync is unsupported, decision D7)
|
||||
action: 'sync' | 'drive-estimator' | 'reextract' | 'ignore';
|
||||
cfi?: string;
|
||||
nextState: RsvpTtsSyncState;
|
||||
}
|
||||
|
||||
export interface RsvpTtsDecisionOptions {
|
||||
// Fixed-layout books can't host the synthetic word stream RSVP sync drives,
|
||||
// so TTS sync is unsupported (decision D7). When set, every position is
|
||||
// dropped without mapping or advancing state.
|
||||
unsupported?: boolean;
|
||||
}
|
||||
|
||||
export const decideRsvpTtsPosition = (
|
||||
state: RsvpTtsSyncState,
|
||||
detail: RsvpTtsPositionDetail,
|
||||
bookKey: string,
|
||||
options: RsvpTtsDecisionOptions = {},
|
||||
): RsvpTtsPositionDecision => {
|
||||
const ignore = (next: RsvpTtsSyncState = state): RsvpTtsPositionDecision => ({
|
||||
action: 'ignore',
|
||||
nextState: next,
|
||||
});
|
||||
|
||||
// D7: never engage on fixed-layout. Checked first so state is left untouched.
|
||||
if (options.unsupported) return ignore();
|
||||
if (detail.bookKey !== bookKey) return ignore();
|
||||
if (!state.following) return ignore();
|
||||
if (typeof detail.cfi !== 'string' || typeof detail.sectionIndex !== 'number') return ignore();
|
||||
|
||||
const sequence = detail.sequence ?? -Infinity;
|
||||
if (sequence <= state.lastSequenceSeen) return ignore();
|
||||
|
||||
// Different section: don't map. Stash the latest sync + bump the sequence,
|
||||
// and let the caller trigger RSVP's re-extract for the new section; the stash
|
||||
// is applied once re-extract completes and the sequence is still current.
|
||||
if (detail.sectionIndex !== state.currentSectionIndex) {
|
||||
return {
|
||||
action: 'reextract',
|
||||
nextState: {
|
||||
...state,
|
||||
lastSequenceSeen: sequence,
|
||||
pendingSync: { cfi: detail.cfi, sequence, sectionIndex: detail.sectionIndex },
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
// Word-level position (Edge): the authoritative path. Mark the engine as
|
||||
// word-capable so subsequent sentence marks don't also drive the estimator.
|
||||
if (detail.kind === 'word') {
|
||||
return {
|
||||
action: 'sync',
|
||||
cfi: detail.cfi,
|
||||
nextState: { ...state, lastSequenceSeen: sequence, hasWordPositions: true },
|
||||
};
|
||||
}
|
||||
|
||||
// Sentence mark. Word-boundary engines emit these too — but once we've seen a
|
||||
// word position, the words drive RSVP and the estimator must stay off (running
|
||||
// it makes RSVP outrun the audio, then word positions snap it back → flashing).
|
||||
if (state.hasWordPositions) {
|
||||
return { action: 'ignore', nextState: { ...state, lastSequenceSeen: sequence } };
|
||||
}
|
||||
return {
|
||||
action: 'drive-estimator',
|
||||
cfi: detail.cfi,
|
||||
nextState: { ...state, lastSequenceSeen: sequence },
|
||||
};
|
||||
};
|
||||
|
||||
// Helper to expand a range to include the full sentence
|
||||
const expandRangeToSentence = (range: Range, doc: Document): Range => {
|
||||
const sentenceRange = doc.createRange();
|
||||
@@ -112,7 +247,10 @@ const expandRangeToSentence = (range: Range, doc: Document): Range => {
|
||||
return range;
|
||||
};
|
||||
|
||||
const RSVPControl: React.FC<RSVPControlProps> = ({ bookKey, gridInsets }) => {
|
||||
const RSVPControl = forwardRef<RSVPControlHandle, RSVPControlProps>(function RSVPControl(
|
||||
{ bookKey, gridInsets },
|
||||
ref,
|
||||
) {
|
||||
const _ = useTranslation();
|
||||
const { envConfig } = useEnv();
|
||||
const settings = useSettingsStore((s) => s.settings);
|
||||
@@ -131,6 +269,20 @@ const RSVPControl: React.FC<RSVPControlProps> = ({ bookKey, gridInsets }) => {
|
||||
const [isActive, setIsActive] = useState(false);
|
||||
const [showStartDialog, setShowStartDialog] = useState(false);
|
||||
const [startChoice, setStartChoice] = useState<RsvpStartChoice | null>(null);
|
||||
// Derived TTS-sync status for the overlay indicator (slice 8b, #3235).
|
||||
const [ttsSyncStatus, setTtsSyncStatus] = useState<TtsSyncStatus>('idle');
|
||||
// True when the last accepted position was sentence-level (non-Edge), so
|
||||
// following is paced by the estimator — the indicator appends " · estimated".
|
||||
const [ttsEstimated, setTtsEstimated] = useState(false);
|
||||
// Whether TTS audio is currently engaged (playing or paused) for this book,
|
||||
// tracked from the tts-playback-state bus. Drives the overlay's audio toggle
|
||||
// active/idle icon (slice 7, #3235).
|
||||
const [ttsActive, setTtsActive] = useState(false);
|
||||
// TTS currently playing (vs paused). Drives the RSVP transport button's icon
|
||||
// when read-along is engaged so play/pause maps to the audio (slice, #3235).
|
||||
const [ttsPlaying, setTtsPlaying] = useState(false);
|
||||
|
||||
useImperativeHandle(ref, () => ({ ttsSyncStatus }), [ttsSyncStatus]);
|
||||
const controllerRef = useRef<RSVPController | null>(null);
|
||||
const tempHighlightRef = useRef<BookNote | null>(null);
|
||||
// renderer.primaryIndex reverts after navigation (paginator #detectPrimaryView),
|
||||
@@ -138,6 +290,29 @@ const RSVPControl: React.FC<RSVPControlProps> = ({ bookKey, gridInsets }) => {
|
||||
const rsvpSectionRef = useRef<number>(-1);
|
||||
const rsvpChapterHrefRef = useRef<string | null>(null);
|
||||
|
||||
// TTS-sync follower state (slice 5, #3235). Mutated by the tts-position /
|
||||
// tts-playback-state handlers and the manual-nav decouple listener.
|
||||
const syncStateRef = useRef<RsvpTtsSyncState>({
|
||||
following: false,
|
||||
lastSequenceSeen: -Infinity,
|
||||
currentSectionIndex: -1,
|
||||
hasWordPositions: false,
|
||||
});
|
||||
// Lets the indicator's "Resume audio" action re-derive the status outside the
|
||||
// sync effect that owns refreshSyncStatus (mirrors the paragraph hook).
|
||||
const refreshSyncStatusRef = useRef<(() => void) | null>(null);
|
||||
|
||||
// Re-engage TTS following after a manual nav decoupled it (indicator action).
|
||||
// Sets following back on and re-derives the status; the next tts-position
|
||||
// event re-syncs. No-op when the controller is gone or sync isn't running.
|
||||
const reengageTtsFollow = useCallback(() => {
|
||||
const sync = syncStateRef.current;
|
||||
if (sync.following) return;
|
||||
sync.following = true;
|
||||
controllerRef.current?.setExternallyDriven(true);
|
||||
refreshSyncStatusRef.current?.();
|
||||
}, []);
|
||||
|
||||
// Helper to remove any existing RSVP highlight
|
||||
const removeRsvpHighlight = useCallback(() => {
|
||||
const view = getView(bookKey);
|
||||
@@ -192,6 +367,248 @@ const RSVPControl: React.FC<RSVPControlProps> = ({ bookKey, gridInsets }) => {
|
||||
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||
}, [bookKey]);
|
||||
|
||||
// Drop TTS following on a user-initiated manual RSVP nav (skip/seek/word-step;
|
||||
// chapter-jump decouples in handleChapterSelect). Stays decoupled until the
|
||||
// next 'playing' re-engages.
|
||||
const decoupleFromTts = useCallback(() => {
|
||||
const sync = syncStateRef.current;
|
||||
if (!sync.following) return;
|
||||
sync.following = false;
|
||||
sync.pendingSync = undefined;
|
||||
const controller = controllerRef.current;
|
||||
controller?.stopEstimator();
|
||||
controller?.setExternallyDriven(false);
|
||||
}, []);
|
||||
|
||||
// Re-extract for the section TTS just drove the view into, then apply the
|
||||
// stashed sync once the view settles on the target section and the stash is
|
||||
// still the latest. Reuses the controller's section-load path
|
||||
// (loadNextPageContent) — the same re-extract the rsvp-request-next-page flow
|
||||
// uses — instead of navigating the view (TTS already navigated it).
|
||||
const reextractForTtsSection = useCallback(() => {
|
||||
const view = getView(bookKey);
|
||||
const controller = controllerRef.current;
|
||||
if (!view || !controller) return;
|
||||
|
||||
const targetSection = syncStateRef.current.pendingSync?.sectionIndex;
|
||||
if (targetSection === undefined) return;
|
||||
|
||||
let cleanup: ReturnType<typeof setTimeout> | null = null;
|
||||
const applyPending = () => {
|
||||
rsvpSectionRef.current = view.renderer.primaryIndex;
|
||||
syncStateRef.current.currentSectionIndex = view.renderer.primaryIndex;
|
||||
const progress = getProgress(bookKey);
|
||||
if (progress?.location) controller.setCurrentCfi(progress.location);
|
||||
controller.loadNextPageContent();
|
||||
|
||||
// Apply the stash only if it's still the latest position and its section
|
||||
// matches what we just extracted (a newer event may have superseded it).
|
||||
const pending = syncStateRef.current.pendingSync;
|
||||
if (
|
||||
pending &&
|
||||
pending.sequence === syncStateRef.current.lastSequenceSeen &&
|
||||
pending.sectionIndex === view.renderer.primaryIndex
|
||||
) {
|
||||
if (syncStateRef.current.pendingSync?.cfi) {
|
||||
controller.syncToCfi(pending.cfi);
|
||||
}
|
||||
syncStateRef.current.pendingSync = undefined;
|
||||
}
|
||||
};
|
||||
|
||||
// If the view is already on the target section, re-extract immediately;
|
||||
// otherwise wait for TTS's own page-follow relocate to land there.
|
||||
if (view.renderer.primaryIndex === targetSection) {
|
||||
applyPending();
|
||||
return;
|
||||
}
|
||||
|
||||
const onRelocate = () => {
|
||||
if (view.renderer.primaryIndex !== targetSection) return; // keep waiting
|
||||
view.removeEventListener('relocate', onRelocate);
|
||||
if (cleanup) clearTimeout(cleanup);
|
||||
applyPending();
|
||||
};
|
||||
view.addEventListener('relocate', onRelocate);
|
||||
cleanup = setTimeout(() => view.removeEventListener('relocate', onRelocate), 5000);
|
||||
}, [bookKey, getProgress, getView]);
|
||||
|
||||
// TTS-sync follower (slice 5, #3235). While an RSVP session is active, follow
|
||||
// the spoken position; TTS is the clock. Filtered to this bookKey, decoupled
|
||||
// by manual nav, re-engaged on the next 'playing'.
|
||||
// Fixed-layout books can't host the synthetic word stream sync drives, so
|
||||
// sync is unsupported there (decision D7): playing never engages and
|
||||
// positions are dropped. The derived ttsSyncStatus feeds the overlay
|
||||
// indicator (slice 8b).
|
||||
useEffect(() => {
|
||||
if (!isActive) return;
|
||||
const controller = controllerRef.current;
|
||||
if (!controller) return;
|
||||
|
||||
const isFixedLayout = !!getBookData(bookKey)?.isFixedLayout;
|
||||
let isPlaying = false;
|
||||
// A TTS session exists (playing OR paused). Distinct from isPlaying so a
|
||||
// pause keeps the indicator + "Audio pace" present (no layout shift) — the
|
||||
// mode is still on; only a full stop clears it.
|
||||
let sessionActive = false;
|
||||
|
||||
const refreshSyncStatus = () => {
|
||||
if (isFixedLayout) {
|
||||
setTtsSyncStatus('unsupported');
|
||||
return;
|
||||
}
|
||||
const sync = syncStateRef.current;
|
||||
if (!sessionActive) {
|
||||
setTtsSyncStatus('idle');
|
||||
} else if (!isPlaying) {
|
||||
// Engaged but paused: keep the indicator visible (mode still on).
|
||||
setTtsSyncStatus('paused');
|
||||
} else if (!sync.following) {
|
||||
// Manual nav decoupled following while TTS still plays.
|
||||
setTtsSyncStatus('decoupled');
|
||||
} else if (sync.pendingSync) {
|
||||
// Cross-section re-extract in flight.
|
||||
setTtsSyncStatus('syncing');
|
||||
} else {
|
||||
setTtsSyncStatus('following');
|
||||
}
|
||||
};
|
||||
refreshSyncStatusRef.current = refreshSyncStatus;
|
||||
refreshSyncStatus();
|
||||
|
||||
const handlePlaybackState = (event: CustomEvent) => {
|
||||
const detail = event.detail as { bookKey?: string; state?: string } | undefined;
|
||||
if (detail?.bookKey !== bookKey) return;
|
||||
const sync = syncStateRef.current;
|
||||
if (detail.state === 'playing') {
|
||||
isPlaying = true;
|
||||
sessionActive = true;
|
||||
setTtsActive(true);
|
||||
setTtsPlaying(true);
|
||||
// D7: never engage following on fixed-layout.
|
||||
if (!isFixedLayout) {
|
||||
// (Re-)engage following from the current section.
|
||||
sync.following = true;
|
||||
sync.currentSectionIndex =
|
||||
rsvpSectionRef.current >= 0
|
||||
? rsvpSectionRef.current
|
||||
: (getView(bookKey)?.renderer.primaryIndex ?? -1);
|
||||
controller.setExternallyDriven(true);
|
||||
}
|
||||
} else if (detail.state === 'paused') {
|
||||
// Paused, still engaged: keep RSVP suspended (frozen on the current
|
||||
// word) so its own timer can't run away while audio is paused. The
|
||||
// transport button (mapped to TTS) resumes it.
|
||||
isPlaying = false;
|
||||
sessionActive = true;
|
||||
setTtsActive(true);
|
||||
setTtsPlaying(false);
|
||||
sync.following = false;
|
||||
sync.pendingSync = undefined;
|
||||
setTtsEstimated(false);
|
||||
controller.stopEstimator();
|
||||
} else if (detail.state === 'stopped') {
|
||||
isPlaying = false;
|
||||
sessionActive = false;
|
||||
setTtsActive(false);
|
||||
setTtsPlaying(false);
|
||||
sync.following = false;
|
||||
sync.pendingSync = undefined;
|
||||
// A full stop may be followed by a voice switch; re-detect word support.
|
||||
sync.hasWordPositions = false;
|
||||
setTtsEstimated(false);
|
||||
controller.stopEstimator();
|
||||
controller.setExternallyDriven(false);
|
||||
}
|
||||
refreshSyncStatus();
|
||||
};
|
||||
|
||||
const handlePosition = (event: CustomEvent) => {
|
||||
const detail = event.detail as RsvpTtsPositionDetail | undefined;
|
||||
if (!detail) return;
|
||||
const decision = decideRsvpTtsPosition(syncStateRef.current, detail, bookKey, {
|
||||
unsupported: isFixedLayout,
|
||||
});
|
||||
syncStateRef.current = decision.nextState;
|
||||
|
||||
switch (decision.action) {
|
||||
case 'sync':
|
||||
// Word-boundary engine: words are authoritative. Kill any estimator
|
||||
// started by the sentence mark that preceded the first word, so the
|
||||
// two never co-run (the cause of jump-ahead-then-snap-back flashing).
|
||||
controller.stopEstimator();
|
||||
if (decision.cfi) controller.syncToCfi(decision.cfi);
|
||||
setTtsEstimated(false);
|
||||
break;
|
||||
case 'drive-estimator': {
|
||||
if (!decision.cfi) break;
|
||||
const viewSettings = getViewSettings(bookKey);
|
||||
const wpm = RSVPController.estimatedWpmFromRate(viewSettings?.ttsRate ?? 1);
|
||||
controller.driveEstimatedFromCfi(decision.cfi, wpm);
|
||||
setTtsEstimated(true);
|
||||
break;
|
||||
}
|
||||
case 'reextract':
|
||||
reextractForTtsSection();
|
||||
break;
|
||||
case 'ignore':
|
||||
default:
|
||||
break;
|
||||
}
|
||||
refreshSyncStatus();
|
||||
};
|
||||
|
||||
const handleManualNav = () => {
|
||||
decoupleFromTts();
|
||||
refreshSyncStatus();
|
||||
};
|
||||
|
||||
eventDispatcher.on('tts-playback-state', handlePlaybackState);
|
||||
eventDispatcher.on('tts-position', handlePosition);
|
||||
controller.addEventListener('rsvp-manual-nav', handleManualNav);
|
||||
|
||||
return () => {
|
||||
eventDispatcher.off('tts-playback-state', handlePlaybackState);
|
||||
eventDispatcher.off('tts-position', handlePosition);
|
||||
controller.removeEventListener('rsvp-manual-nav', handleManualNav);
|
||||
// Leaving sync: restore the controller's own pacing.
|
||||
syncStateRef.current.following = false;
|
||||
syncStateRef.current.pendingSync = undefined;
|
||||
controller.stopEstimator();
|
||||
controller.setExternallyDriven(false);
|
||||
setTtsSyncStatus('idle');
|
||||
setTtsEstimated(false);
|
||||
setTtsActive(false);
|
||||
setTtsPlaying(false);
|
||||
};
|
||||
}, [
|
||||
isActive,
|
||||
bookKey,
|
||||
getBookData,
|
||||
getView,
|
||||
getViewSettings,
|
||||
decoupleFromTts,
|
||||
reextractForTtsSection,
|
||||
]);
|
||||
|
||||
// One-time-per-session decouple toast: the first time following drops while
|
||||
// TTS still plays, tell the user once. Reset when following re-engages so a
|
||||
// later decouple notifies again.
|
||||
const decoupleToastShownRef = useRef(false);
|
||||
useEffect(() => {
|
||||
if (ttsSyncStatus === 'decoupled') {
|
||||
if (!decoupleToastShownRef.current) {
|
||||
decoupleToastShownRef.current = true;
|
||||
eventDispatcher.dispatch('toast', {
|
||||
message: _('Stopped following audio'),
|
||||
type: 'info',
|
||||
});
|
||||
}
|
||||
} else if (ttsSyncStatus === 'following') {
|
||||
decoupleToastShownRef.current = false;
|
||||
}
|
||||
}, [ttsSyncStatus, _]);
|
||||
|
||||
const handleStart = useCallback(
|
||||
(selectionText?: string) => {
|
||||
// RSVP can be started from the menu or the keyboard shortcut (#4473);
|
||||
@@ -463,6 +880,9 @@ const RSVPControl: React.FC<RSVPControlProps> = ({ bookKey, gridInsets }) => {
|
||||
const view = getView(bookKey);
|
||||
if (!view) return;
|
||||
|
||||
// A chapter jump is a user-initiated nav: decouple from TTS following.
|
||||
decoupleFromTts();
|
||||
|
||||
const onRelocate = (e: Event) => {
|
||||
view.removeEventListener('relocate', onRelocate);
|
||||
const detail = (e as CustomEvent).detail as { section?: PageInfo; tocItem?: TOCItem };
|
||||
@@ -480,7 +900,7 @@ const RSVPControl: React.FC<RSVPControlProps> = ({ bookKey, gridInsets }) => {
|
||||
view.addEventListener('relocate', onRelocate);
|
||||
view.goTo(href);
|
||||
},
|
||||
[bookKey, getProgress, getView],
|
||||
[bookKey, getProgress, getView, decoupleFromTts],
|
||||
);
|
||||
|
||||
const handleRequestNextPage = useCallback(async () => {
|
||||
@@ -546,11 +966,54 @@ const RSVPControl: React.FC<RSVPControlProps> = ({ bookKey, gridInsets }) => {
|
||||
// Book language drives dictionary provider selection for context lookups (#4475).
|
||||
const dictionaryLang = bookData?.bookDoc?.metadata?.language as string | undefined;
|
||||
const handleManageDictionary = useCallback(() => {
|
||||
// Open dictionary management OVER the RSVP overlay (RSVP stays open). The
|
||||
// settings dialog is raised above the overlay's z-[10000] (see SettingsDialog),
|
||||
// and RSVP's capture-phase keyboard handler bails while it's open so the
|
||||
// settings inputs / Escape work (see RSVPOverlay).
|
||||
setSettingsDialogBookKey(bookKey);
|
||||
setActiveSettingsItemId('settings.language.dictionaries.manage');
|
||||
setSettingsDialogOpen(true);
|
||||
}, [bookKey, setActiveSettingsItemId, setSettingsDialogBookKey, setSettingsDialogOpen]);
|
||||
|
||||
// Audio (TTS) toggle from the overlay (slice 7, decision 5, #3235). When TTS
|
||||
// is engaged, stop it; otherwise start it from the displayed RSVP word with
|
||||
// start-alignment — the word's range (validated live) + its section index — so
|
||||
// audio begins at the same word RSVP is flashing.
|
||||
const handleToggleTtsAudio = useCallback(() => {
|
||||
if (ttsActive) {
|
||||
eventDispatcher.dispatch('tts-stop', { bookKey });
|
||||
return;
|
||||
}
|
||||
const controller = controllerRef.current;
|
||||
const currentWord = controller?.currentDisplayWord ?? null;
|
||||
const view = getView(bookKey);
|
||||
const currentDoc =
|
||||
typeof currentWord?.docIndex === 'number'
|
||||
? view?.renderer.getContents().find((c) => c.index === currentWord.docIndex)?.doc
|
||||
: undefined;
|
||||
const detail = buildRsvpTtsSpeakDetail(currentWord, bookKey, currentDoc);
|
||||
eventDispatcher.dispatch('tts-speak', detail ?? { bookKey });
|
||||
}, [bookKey, getView, ttsActive]);
|
||||
|
||||
// RSVP transport (center play/pause) mapped to TTS play/pause while read-along
|
||||
// is engaged (#3235): the button should pause/resume the audio, not RSVP's own
|
||||
// (suspended) timer. Reuses the existing tts-toggle-play bus event.
|
||||
const handleToggleTtsPlay = useCallback(() => {
|
||||
eventDispatcher.dispatch('tts-toggle-play', { bookKey });
|
||||
}, [bookKey]);
|
||||
|
||||
// Change the TTS playback rate from the overlay's rate picker (decision 6).
|
||||
// The TTS rate panel is unreachable behind the full-screen overlay, so dispatch
|
||||
// a one-shot tts-set-rate the TTS hook applies via its existing rate path.
|
||||
const handleSetTtsRate = useCallback(
|
||||
(rate: number) => {
|
||||
eventDispatcher.dispatch('tts-set-rate', { bookKey, rate });
|
||||
},
|
||||
[bookKey],
|
||||
);
|
||||
|
||||
const ttsRate = viewSettings?.ttsRate ?? 1;
|
||||
|
||||
// Use portal to render overlay at body level to avoid stacking context issues
|
||||
const portalContainer = typeof document !== 'undefined' ? document.body : null;
|
||||
|
||||
@@ -581,6 +1044,15 @@ const RSVPControl: React.FC<RSVPControlProps> = ({ bookKey, gridInsets }) => {
|
||||
currentChapterHref={currentChapterHref}
|
||||
fontFamily={fontFamily}
|
||||
lang={dictionaryLang}
|
||||
ttsSyncStatus={ttsSyncStatus}
|
||||
estimated={ttsEstimated}
|
||||
ttsActive={ttsActive}
|
||||
ttsPlaying={ttsPlaying}
|
||||
ttsRate={ttsRate}
|
||||
onToggleTtsAudio={handleToggleTtsAudio}
|
||||
onToggleTtsPlay={handleToggleTtsPlay}
|
||||
onSetTtsRate={handleSetTtsRate}
|
||||
onResumeTtsFollow={reengageTtsFollow}
|
||||
onClose={handleClose}
|
||||
onChapterSelect={handleChapterSelect}
|
||||
onRequestNextPage={handleRequestNextPage}
|
||||
@@ -590,6 +1062,6 @@ const RSVPControl: React.FC<RSVPControlProps> = ({ bookKey, gridInsets }) => {
|
||||
)}
|
||||
</>
|
||||
);
|
||||
};
|
||||
});
|
||||
|
||||
export default RSVPControl;
|
||||
|
||||
@@ -6,6 +6,7 @@ import { Insets } from '@/types/misc';
|
||||
import { RsvpState, RSVPController } from '@/services/rsvp';
|
||||
import { containsCJK } from '@/services/rsvp/utils';
|
||||
import { useThemeStore } from '@/store/themeStore';
|
||||
import { useSettingsStore } from '@/store/settingsStore';
|
||||
import { TOCItem } from '@/libs/document';
|
||||
import {
|
||||
IoClose,
|
||||
@@ -20,12 +21,16 @@ import {
|
||||
IoChevronDown,
|
||||
IoSettingsSharp,
|
||||
IoSearch,
|
||||
IoVolumeHigh,
|
||||
IoVolumeMediumOutline,
|
||||
IoLockClosed,
|
||||
} from 'react-icons/io5';
|
||||
import { useTranslation } from '@/hooks/useTranslation';
|
||||
import { getPopupPosition, Position } from '@/utils/sel';
|
||||
import { Overlay } from '@/components/Overlay';
|
||||
import DictionarySheet from '@/app/reader/components/annotator/DictionarySheet';
|
||||
import DictionaryPopup from '@/app/reader/components/annotator/DictionaryPopup';
|
||||
import TTSFollowIndicator, { TtsSyncStatus } from '@/app/reader/components/tts/TTSFollowIndicator';
|
||||
|
||||
interface FlatChapter {
|
||||
label: string;
|
||||
@@ -79,6 +84,10 @@ const CONTEXT_CHUNK_SIZE = 50;
|
||||
const CONTEXT_WINDOW_BEFORE = 200;
|
||||
const CONTEXT_WINDOW_AFTER = 1000;
|
||||
|
||||
// TTS rate options for the overlay's rate picker (decision 6) — mirrors the
|
||||
// 0.5–3.0 range the TTS panel slider clamps to, in 0.25 steps.
|
||||
const TTS_RATE_OPTIONS = [0.5, 0.75, 1.0, 1.25, 1.5, 1.75, 2.0, 2.25, 2.5, 2.75, 3.0];
|
||||
|
||||
// Dictionary lookup popup sizing (mirrors the reader's Annotator popup).
|
||||
const DICT_POPUP_PADDING = 10;
|
||||
const DICT_POPUP_MAX_WIDTH = 480;
|
||||
@@ -97,6 +106,24 @@ interface RSVPOverlayProps {
|
||||
fontFamily?: string;
|
||||
/** Book language, used to pick dictionary providers for context lookups. */
|
||||
lang?: string;
|
||||
/** Derived TTS-sync status driving the "following audio" indicator (#3235). */
|
||||
ttsSyncStatus?: TtsSyncStatus;
|
||||
/** True when following is paced by the estimator (non-Edge sentence sync). */
|
||||
estimated?: boolean;
|
||||
/** True when TTS audio is engaged (playing/paused) — drives the audio toggle. */
|
||||
ttsActive?: boolean;
|
||||
/** True when TTS is actively playing (vs paused) — drives the transport icon. */
|
||||
ttsPlaying?: boolean;
|
||||
/** Current TTS playback rate, shown selected in the rate picker (decision 6). */
|
||||
ttsRate?: number;
|
||||
/** Toggle TTS audio: start from the current word, or stop when engaged. */
|
||||
onToggleTtsAudio?: () => void;
|
||||
/** Pause/resume TTS — the transport play/pause maps here while read-along is on. */
|
||||
onToggleTtsPlay?: () => void;
|
||||
/** Set the TTS rate (one-shot) when the WPM control is TTS-driven. */
|
||||
onSetTtsRate?: (rate: number) => void;
|
||||
/** Re-engage following after a manual nav decoupled it (indicator action). */
|
||||
onResumeTtsFollow?: () => void;
|
||||
onClose: () => void;
|
||||
onChapterSelect: (href: string) => void;
|
||||
onRequestNextPage: () => void;
|
||||
@@ -111,6 +138,15 @@ const RSVPOverlay: React.FC<RSVPOverlayProps> = ({
|
||||
currentChapterHref,
|
||||
fontFamily,
|
||||
lang,
|
||||
ttsSyncStatus = 'idle',
|
||||
estimated = false,
|
||||
ttsActive = false,
|
||||
ttsPlaying = false,
|
||||
ttsRate = 1,
|
||||
onToggleTtsAudio,
|
||||
onToggleTtsPlay,
|
||||
onSetTtsRate,
|
||||
onResumeTtsFollow,
|
||||
onClose,
|
||||
onChapterSelect,
|
||||
onRequestNextPage,
|
||||
@@ -118,12 +154,23 @@ const RSVPOverlay: React.FC<RSVPOverlayProps> = ({
|
||||
}) => {
|
||||
const _ = useTranslation();
|
||||
const { themeCode, isDarkMode: _isDarkMode } = useThemeStore();
|
||||
const isSettingsDialogOpen = useSettingsStore((s) => s.isSettingsDialogOpen);
|
||||
const [state, setState] = useState<RsvpState>(controller.currentState);
|
||||
const currentWord = controller.currentDisplayWord;
|
||||
// The transport (center) play/pause controls TTS while read-along is engaged,
|
||||
// otherwise RSVP's own timer (#3235). A ref keeps the latest closure so the
|
||||
// capture-phase keyboard/tap effects don't need it in their dep arrays.
|
||||
const transportToggleRef = useRef<() => void>(() => {});
|
||||
transportToggleRef.current = () => {
|
||||
if (ttsActive && onToggleTtsPlay) onToggleTtsPlay();
|
||||
else controller.togglePlayPause();
|
||||
};
|
||||
const transportPlaying = ttsActive ? ttsPlaying : state.playing;
|
||||
const [countdown, setCountdown] = useState<number | null>(controller.currentCountdown);
|
||||
const [showChapterDropdown, setShowChapterDropdown] = useState(false);
|
||||
const chapterDropdownRef = useRef<HTMLDivElement>(null);
|
||||
const [showWpmDropdown, setShowWpmDropdown] = useState(false);
|
||||
const [showRateDropdown, setShowRateDropdown] = useState(false);
|
||||
const [showSettings, setShowSettings] = useState(false);
|
||||
const [contextCollapsed, setContextCollapsed] = useState(() => {
|
||||
try {
|
||||
@@ -236,12 +283,15 @@ const RSVPOverlay: React.FC<RSVPOverlayProps> = ({
|
||||
// While the dictionary is open it owns the keyboard (e.g. Escape closes
|
||||
// the dictionary, not the whole RSVP session).
|
||||
if (dict) return;
|
||||
// Dictionary management (settings dialog) opens OVER RSVP; let it own the
|
||||
// keyboard so its inputs accept space and Escape closes it, not RSVP.
|
||||
if (isSettingsDialogOpen) return;
|
||||
|
||||
switch (event.key) {
|
||||
case ' ':
|
||||
event.preventDefault();
|
||||
event.stopPropagation();
|
||||
controller.togglePlayPause();
|
||||
transportToggleRef.current();
|
||||
break;
|
||||
case 'Escape':
|
||||
event.preventDefault();
|
||||
@@ -292,7 +342,7 @@ const RSVPOverlay: React.FC<RSVPOverlayProps> = ({
|
||||
// Use capture phase to handle events before they reach dropdown/select elements
|
||||
document.addEventListener('keydown', handleKeyboard, { capture: true });
|
||||
return () => document.removeEventListener('keydown', handleKeyboard, { capture: true });
|
||||
}, [state.active, controller, onClose, dict]);
|
||||
}, [state.active, controller, onClose, dict, isSettingsDialogOpen]);
|
||||
|
||||
const effectiveChapterHref = currentChapterHref;
|
||||
|
||||
@@ -452,7 +502,7 @@ const RSVPOverlay: React.FC<RSVPOverlayProps> = ({
|
||||
} else if (tapX > screenWidth * 0.75) {
|
||||
controller.skipForward(15);
|
||||
} else {
|
||||
controller.togglePlayPause();
|
||||
transportToggleRef.current();
|
||||
}
|
||||
}
|
||||
};
|
||||
@@ -620,6 +670,13 @@ const RSVPOverlay: React.FC<RSVPOverlayProps> = ({
|
||||
const currentFontSize =
|
||||
FONT_SIZE_OPTIONS[fontSizeIndex] ?? FONT_SIZE_OPTIONS[DEFAULT_FONT_SIZE_INDEX]!;
|
||||
|
||||
// The WPM timer doesn't drive pacing while RSVP follows TTS — the voice does.
|
||||
// Replace the WPM control with an "Audio pace" affordance that opens a TTS
|
||||
// rate picker instead (decision 6, #3235).
|
||||
// 'paused' keeps the WPM "Audio pace" lock too, so pausing doesn't shift layout.
|
||||
const ttsDriven =
|
||||
ttsSyncStatus === 'following' || ttsSyncStatus === 'syncing' || ttsSyncStatus === 'paused';
|
||||
|
||||
return (
|
||||
<div
|
||||
data-testid='rsvp-overlay'
|
||||
@@ -697,31 +754,57 @@ const RSVPOverlay: React.FC<RSVPOverlayProps> = ({
|
||||
)}
|
||||
</div>
|
||||
|
||||
{/* WPM selector */}
|
||||
{/* WPM selector — while RSVP follows TTS the timer no longer paces, so it
|
||||
becomes an "Audio pace" affordance that opens a TTS rate picker
|
||||
instead (decision 6). aria-disabled (not hard-disabled) keeps it
|
||||
focusable as a hint; the lock glyph + border reads in e-ink without
|
||||
relying on opacity. */}
|
||||
<div className='relative shrink-0'>
|
||||
<button
|
||||
className='flex items-center gap-1 rounded-full border border-gray-500/20 bg-gray-500/10 px-3 py-1.5 text-sm tabular-nums transition-colors hover:bg-gray-500/20'
|
||||
onClick={() => setShowWpmDropdown(!showWpmDropdown)}
|
||||
aria-label={_('Select reading speed')}
|
||||
title={_('Select reading speed')}
|
||||
>
|
||||
<span className='font-semibold'>{state.wpm}</span>
|
||||
<span className='ml-0.5 text-xs opacity-50'>WPM</span>
|
||||
<svg
|
||||
viewBox='0 0 24 24'
|
||||
fill='none'
|
||||
stroke='currentColor'
|
||||
strokeWidth='2.5'
|
||||
className='ml-0.5 h-3 w-3 shrink-0 opacity-50'
|
||||
{ttsDriven ? (
|
||||
<button
|
||||
className='eink-bordered flex items-center gap-1.5 rounded-full border border-gray-500/20 bg-gray-500/10 px-3 py-1.5 text-sm transition-colors hover:bg-gray-500/20'
|
||||
onClick={() => setShowRateDropdown(!showRateDropdown)}
|
||||
aria-disabled='true'
|
||||
aria-label={_('Audio pace')}
|
||||
title={_('Speed follows audio')}
|
||||
>
|
||||
<path d='M6 9l6 6 6-6' />
|
||||
</svg>
|
||||
</button>
|
||||
{showWpmDropdown && (
|
||||
<IoLockClosed className='h-3.5 w-3.5 shrink-0 opacity-70' aria-hidden='true' />
|
||||
<span className='font-medium'>{_('Audio pace')}</span>
|
||||
<svg
|
||||
viewBox='0 0 24 24'
|
||||
fill='none'
|
||||
stroke='currentColor'
|
||||
strokeWidth='2.5'
|
||||
className='ms-0.5 h-3 w-3 shrink-0 opacity-50'
|
||||
>
|
||||
<path d='M6 9l6 6 6-6' />
|
||||
</svg>
|
||||
</button>
|
||||
) : (
|
||||
<button
|
||||
className='flex items-center gap-1 rounded-full border border-gray-500/20 bg-gray-500/10 px-3 py-1.5 text-sm tabular-nums transition-colors hover:bg-gray-500/20'
|
||||
onClick={() => setShowWpmDropdown(!showWpmDropdown)}
|
||||
aria-label={_('Select reading speed')}
|
||||
title={_('Select reading speed')}
|
||||
>
|
||||
<span className='font-semibold'>{state.wpm}</span>
|
||||
<span className='ms-0.5 text-xs opacity-50'>WPM</span>
|
||||
<svg
|
||||
viewBox='0 0 24 24'
|
||||
fill='none'
|
||||
stroke='currentColor'
|
||||
strokeWidth='2.5'
|
||||
className='ms-0.5 h-3 w-3 shrink-0 opacity-50'
|
||||
>
|
||||
<path d='M6 9l6 6 6-6' />
|
||||
</svg>
|
||||
</button>
|
||||
)}
|
||||
{showWpmDropdown && !ttsDriven && (
|
||||
<>
|
||||
<Overlay onDismiss={() => setShowWpmDropdown(false)} />
|
||||
<div
|
||||
className='absolute right-0 top-full z-[100] mt-1.5 max-h-64 min-w-[7rem] overflow-y-auto rounded-2xl border border-gray-500/20 shadow-2xl'
|
||||
className='absolute end-0 top-full z-[100] mt-1.5 max-h-64 min-w-[7rem] overflow-y-auto rounded-2xl border border-gray-500/20 shadow-2xl'
|
||||
style={{ backgroundColor: bgColor }}
|
||||
>
|
||||
{controller.getWpmOptions().map((wpm) => (
|
||||
@@ -744,9 +827,53 @@ const RSVPOverlay: React.FC<RSVPOverlayProps> = ({
|
||||
</div>
|
||||
</>
|
||||
)}
|
||||
{showRateDropdown && ttsDriven && (
|
||||
<>
|
||||
<Overlay onDismiss={() => setShowRateDropdown(false)} />
|
||||
<div
|
||||
className='absolute end-0 top-full z-[100] mt-1.5 max-h-64 min-w-[7rem] overflow-y-auto rounded-2xl border border-gray-500/20 shadow-2xl'
|
||||
style={{ backgroundColor: bgColor }}
|
||||
>
|
||||
{TTS_RATE_OPTIONS.map((rate) => (
|
||||
<button
|
||||
key={rate}
|
||||
className={clsx(
|
||||
'flex w-full items-center justify-between gap-3 whitespace-nowrap rounded-md border-none bg-transparent px-4 py-1.5 text-sm tabular-nums transition-colors first:rounded-t-2xl last:rounded-b-2xl hover:bg-gray-500/15',
|
||||
Math.abs(ttsRate - rate) < 0.001 &&
|
||||
'bg-[color-mix(in_srgb,var(--rsvp-accent)_15%,transparent)] font-semibold',
|
||||
)}
|
||||
onClick={() => {
|
||||
onSetTtsRate?.(rate);
|
||||
setShowRateDropdown(false);
|
||||
}}
|
||||
>
|
||||
<span>{rate.toFixed(2)}×</span>
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
</>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* TTS "following audio" status row — slim, below the header and above the
|
||||
context panel (never inside the transport row). Uses the 'plain' variant
|
||||
to match the overlay's own theme-painted surface. idle/unsupported
|
||||
collapse to nothing. */}
|
||||
{(ttsSyncStatus === 'following' ||
|
||||
ttsSyncStatus === 'syncing' ||
|
||||
ttsSyncStatus === 'decoupled' ||
|
||||
ttsSyncStatus === 'paused') && (
|
||||
<div className='flex shrink-0 justify-center px-3 pb-1 md:px-4'>
|
||||
<TTSFollowIndicator
|
||||
status={ttsSyncStatus}
|
||||
estimated={estimated}
|
||||
onResume={onResumeTtsFollow}
|
||||
variant='plain'
|
||||
/>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Context panel (always visible, collapsible) */}
|
||||
<div className='mx-3 overflow-hidden rounded-lg border border-gray-500/20 bg-gray-500/10 md:mx-4 md:rounded-xl'>
|
||||
<button
|
||||
@@ -974,15 +1101,15 @@ const RSVPOverlay: React.FC<RSVPOverlayProps> = ({
|
||||
</button>
|
||||
|
||||
<button
|
||||
aria-label={state.playing ? _('Pause') : _('Play')}
|
||||
aria-label={transportPlaying ? _('Pause') : _('Play')}
|
||||
className={clsx(
|
||||
'flex h-14 w-14 cursor-pointer items-center justify-center rounded-full border-none bg-gray-500/15 transition-colors hover:bg-gray-500/25 active:scale-95 md:h-16 md:w-16',
|
||||
state.playing ? '' : 'ps-1',
|
||||
transportPlaying ? '' : 'ps-1',
|
||||
)}
|
||||
onClick={() => controller.togglePlayPause()}
|
||||
title={state.playing ? _('Pause (Space)') : _('Play (Space)')}
|
||||
onClick={() => transportToggleRef.current()}
|
||||
title={transportPlaying ? _('Pause (Space)') : _('Play (Space)')}
|
||||
>
|
||||
{state.playing ? (
|
||||
{transportPlaying ? (
|
||||
<IoPause className='h-7 w-7 md:h-8 md:w-8' />
|
||||
) : (
|
||||
<IoPlay className='h-7 w-7 md:h-8 md:w-8' />
|
||||
@@ -1017,17 +1144,48 @@ const RSVPOverlay: React.FC<RSVPOverlayProps> = ({
|
||||
<span className='text-xs font-semibold opacity-80'>15</span>
|
||||
</button>
|
||||
|
||||
<button
|
||||
aria-label={_('Settings')}
|
||||
className={clsx(
|
||||
'absolute right-0 flex h-9 w-9 cursor-pointer items-center justify-center rounded-full border-none bg-transparent transition-colors hover:bg-gray-500/20 active:scale-95',
|
||||
showSettings && 'bg-gray-500/15',
|
||||
)}
|
||||
onClick={() => setShowSettings((prev) => !prev)}
|
||||
title={_('Settings')}
|
||||
>
|
||||
<IoSettingsSharp className='h-4 w-4 md:h-5 md:w-5' />
|
||||
</button>
|
||||
{/* Trailing cluster: audio (TTS) toggle + divider + settings gear.
|
||||
The audio toggle starts TTS from the displayed word (or stops it
|
||||
when engaged) — never a second play triangle (decision 5). Active
|
||||
state uses a filled glyph + eink-bordered surface so it reads in
|
||||
e-ink without relying on color. */}
|
||||
<div className='absolute end-0 flex items-center gap-1'>
|
||||
<button
|
||||
aria-label={ttsActive ? _('Pause audio') : _('Play audio')}
|
||||
className={clsx(
|
||||
'touch-target flex h-9 w-9 cursor-pointer items-center justify-center rounded-full border-none transition-colors active:scale-95',
|
||||
ttsActive
|
||||
? 'eink-bordered bg-[color-mix(in_srgb,var(--rsvp-accent)_18%,transparent)]'
|
||||
: 'bg-transparent hover:bg-gray-500/20',
|
||||
)}
|
||||
onClick={() => onToggleTtsAudio?.()}
|
||||
title={ttsActive ? _('Pause audio') : _('Play audio')}
|
||||
>
|
||||
{ttsActive ? (
|
||||
<IoVolumeHigh
|
||||
className='h-4 w-4 md:h-5 md:w-5'
|
||||
style={{ color: accentColor }}
|
||||
aria-hidden='true'
|
||||
/>
|
||||
) : (
|
||||
<IoVolumeMediumOutline className='h-4 w-4 md:h-5 md:w-5' aria-hidden='true' />
|
||||
)}
|
||||
</button>
|
||||
|
||||
<span className='h-5 w-px bg-gray-500/30' aria-hidden='true' />
|
||||
|
||||
<button
|
||||
aria-label={_('Settings')}
|
||||
className={clsx(
|
||||
'flex h-9 w-9 cursor-pointer items-center justify-center rounded-full border-none bg-transparent transition-colors hover:bg-gray-500/20 active:scale-95',
|
||||
showSettings && 'bg-gray-500/15',
|
||||
)}
|
||||
onClick={() => setShowSettings((prev) => !prev)}
|
||||
title={_('Settings')}
|
||||
>
|
||||
<IoSettingsSharp className='h-4 w-4 md:h-5 md:w-5' />
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Settings row (collapsible) */}
|
||||
|
||||
@@ -0,0 +1,45 @@
|
||||
import { RsvpWord } from '@/services/rsvp';
|
||||
|
||||
// Detail payload for the app-bus `tts-speak` event (see useTTSControl.handleTTSSpeak,
|
||||
// which honors a passed `range` + `index`). RSVP starts audio from the displayed
|
||||
// word so the listener and the flashing word stay aligned.
|
||||
export interface RsvpTtsSpeakDetail {
|
||||
bookKey: string;
|
||||
// Section spine index of the displayed word — starts TTS in the right section.
|
||||
index?: number;
|
||||
// Live DOM range of the displayed word — starts TTS at the exact word. Omitted
|
||||
// when the word has no range or the range is stale (its document no longer
|
||||
// matches the current content), so TTS falls back to its own start position.
|
||||
range?: Range;
|
||||
}
|
||||
|
||||
// Build the `tts-speak` detail for "start audio from the current RSVP word"
|
||||
// (decision 5, #3235). Returns null when there is no current word to align to.
|
||||
//
|
||||
// Start-alignment rules:
|
||||
// - index = the word's spine index (when known), so audio begins in the
|
||||
// displayed section even if the range can't be used.
|
||||
// - range is included ONLY when it is live: it exists and its ownerDocument is
|
||||
// the document RSVP is currently rendering (`currentDoc`). A stale or
|
||||
// cross-document range would resolve to the wrong place, so it is dropped and
|
||||
// TTS falls back to its own start position.
|
||||
export const buildRsvpTtsSpeakDetail = (
|
||||
currentWord: RsvpWord | null | undefined,
|
||||
bookKey: string,
|
||||
currentDoc: Document | null | undefined,
|
||||
): RsvpTtsSpeakDetail | null => {
|
||||
if (!currentWord) return null;
|
||||
|
||||
const detail: RsvpTtsSpeakDetail = { bookKey };
|
||||
|
||||
if (typeof currentWord.docIndex === 'number') {
|
||||
detail.index = currentWord.docIndex;
|
||||
}
|
||||
|
||||
const range = currentWord.range;
|
||||
if (range && currentDoc && range.startContainer.ownerDocument === currentDoc) {
|
||||
detail.range = range;
|
||||
}
|
||||
|
||||
return detail;
|
||||
};
|
||||
@@ -0,0 +1,117 @@
|
||||
'use client';
|
||||
|
||||
import React from 'react';
|
||||
import clsx from 'clsx';
|
||||
import { IoVolumeHigh, IoSync } from 'react-icons/io5';
|
||||
import { useTranslation } from '@/hooks/useTranslation';
|
||||
|
||||
// The derived states both reader modes expose (slice 5, #3235). idle and
|
||||
// unsupported render nothing — we never nag, especially not on fixed-layout.
|
||||
// 'paused' keeps the pill visible while TTS is engaged but paused (no layout
|
||||
// shift) — it renders like 'following'.
|
||||
export type TtsSyncStatus =
|
||||
| 'idle'
|
||||
| 'following'
|
||||
| 'syncing'
|
||||
| 'decoupled'
|
||||
| 'paused'
|
||||
| 'unsupported';
|
||||
|
||||
interface TTSFollowIndicatorProps {
|
||||
status: TtsSyncStatus;
|
||||
/**
|
||||
* RSVP non-Edge sentence-level following is paced by an estimator, not exact
|
||||
* word marks, so the label is qualified with " · estimated".
|
||||
*/
|
||||
estimated?: boolean;
|
||||
/** Re-engage following. Only wired through on the decoupled (action) state. */
|
||||
onResume?: () => void;
|
||||
/**
|
||||
* Surface idiom. 'base' (default) uses daisyui base tokens — for the paragraph
|
||||
* overlay, where the global [data-eink] rules apply. 'plain' uses the neutral
|
||||
* gray-500 overlay + currentColor text to match the RSVP overlay's own
|
||||
* theme-painted surface (the same idiom as its chapter/WPM pills and "Look up"
|
||||
* pill), where daisyui base tokens would clash with the book theme.
|
||||
*/
|
||||
variant?: 'base' | 'plain';
|
||||
className?: string;
|
||||
}
|
||||
|
||||
// Shared pill chassis. eink-bordered is applied unconditionally: in eink mode it
|
||||
// swaps to bg-base-100 + a 1px base-content border, and the RSVP overlay paints
|
||||
// its own theme surface where the global [data-eink] rules don't auto-apply
|
||||
// (mirrors the RSVP "Look up" pill). Glyph + text together so the meaning never
|
||||
// rests on color alone — required to read in e-ink monochrome.
|
||||
const PILL_BASE =
|
||||
'eink-bordered inline-flex items-center gap-1.5 rounded-full px-3 py-1 text-xs font-medium leading-none whitespace-nowrap';
|
||||
|
||||
// Per-variant fills. 'plain' inherits text color (currentColor) so it reads on
|
||||
// any book theme the RSVP overlay paints.
|
||||
const STATUS_FILL: Record<NonNullable<TTSFollowIndicatorProps['variant']>, string> = {
|
||||
base: 'bg-base-content/10 text-base-content',
|
||||
plain: 'bg-gray-500/15',
|
||||
};
|
||||
const ACTION_FILL: Record<NonNullable<TTSFollowIndicatorProps['variant']>, string> = {
|
||||
base: 'bg-base-200 text-base-content transition-colors hover:bg-base-300',
|
||||
plain: 'bg-gray-500/20 transition-colors hover:bg-gray-500/30',
|
||||
};
|
||||
|
||||
const TTSFollowIndicator: React.FC<TTSFollowIndicatorProps> = ({
|
||||
status,
|
||||
estimated = false,
|
||||
onResume,
|
||||
variant = 'base',
|
||||
className,
|
||||
}) => {
|
||||
const _ = useTranslation();
|
||||
|
||||
// Never nag: idle has nothing to follow, unsupported (fixed-layout) can never
|
||||
// engage. Both render null rather than a placeholder.
|
||||
if (status === 'idle' || status === 'unsupported') return null;
|
||||
|
||||
if (status === 'decoupled') {
|
||||
// The only actionable state — the whole pill is the button. touch-target
|
||||
// extends the hit area to 44px on mobile without growing the rendered pill.
|
||||
return (
|
||||
<button
|
||||
type='button'
|
||||
aria-label={_('Resume audio')}
|
||||
onClick={onResume}
|
||||
className={clsx(
|
||||
PILL_BASE,
|
||||
'touch-target',
|
||||
ACTION_FILL[variant],
|
||||
'focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-base-content/15',
|
||||
className,
|
||||
)}
|
||||
>
|
||||
<IoSync className='h-3.5 w-3.5 shrink-0' aria-hidden='true' />
|
||||
<span>{_('Resume audio')}</span>
|
||||
</button>
|
||||
);
|
||||
}
|
||||
|
||||
// following / syncing: a filled, non-interactive status pill. syncing adds the
|
||||
// loading-dots affordance while a cross-section re-extract is in flight.
|
||||
return (
|
||||
<div
|
||||
role='status'
|
||||
aria-live='polite'
|
||||
className={clsx(PILL_BASE, STATUS_FILL[variant], className)}
|
||||
>
|
||||
{status === 'syncing' ? (
|
||||
<span
|
||||
className='loading loading-dots loading-xs shrink-0'
|
||||
aria-hidden='true'
|
||||
data-testid='tts-follow-syncing'
|
||||
/>
|
||||
) : (
|
||||
<IoVolumeHigh className='h-3.5 w-3.5 shrink-0' aria-hidden='true' />
|
||||
)}
|
||||
<span>{_('Following audio')}</span>
|
||||
{estimated && <span className='not-eink:opacity-70'>{_(' · estimated')}</span>}
|
||||
</div>
|
||||
);
|
||||
};
|
||||
|
||||
export default TTSFollowIndicator;
|
||||
@@ -1,5 +1,6 @@
|
||||
import { useCallback, useEffect, useRef, useState } from 'react';
|
||||
import { useReaderStore } from '@/store/readerStore';
|
||||
import { useBookDataStore } from '@/store/bookDataStore';
|
||||
import { useEnv } from '@/context/EnvContext';
|
||||
import { FoliateView } from '@/types/view';
|
||||
import { eventDispatcher } from '@/utils/event';
|
||||
@@ -7,12 +8,32 @@ import { saveViewSettings } from '@/helpers/settings';
|
||||
import { ParagraphIterator } from '@/utils/paragraph';
|
||||
import { getParagraphPresentation } from '@/utils/paragraphPresentation';
|
||||
import { DEFAULT_PARAGRAPH_MODE_CONFIG } from '@/services/constants';
|
||||
import { isRangeLike } from '@/utils/range';
|
||||
import {
|
||||
buildParagraphTtsSpeakDetail,
|
||||
computeParagraphHighlightOffsets,
|
||||
decideParagraphTtsHighlight,
|
||||
} from '@/app/reader/components/paragraph/paragraphTts';
|
||||
|
||||
interface UseParagraphModeProps {
|
||||
bookKey: string;
|
||||
viewRef: React.RefObject<FoliateView | null>;
|
||||
}
|
||||
|
||||
// Derived state for the TTS-sync indicator (later slice renders it).
|
||||
// - 'unsupported': fixed-layout book; sync can never engage.
|
||||
// - 'idle': TTS not playing / not engaged.
|
||||
// - 'following': actively following the spoken position.
|
||||
// - 'syncing': a cross-section position is pending a re-init.
|
||||
// - 'decoupled': was following, user took manual control (TTS still playing).
|
||||
export type TtsSyncStatus =
|
||||
| 'unsupported'
|
||||
| 'idle'
|
||||
| 'following'
|
||||
| 'syncing'
|
||||
| 'decoupled'
|
||||
| 'paused';
|
||||
|
||||
export interface ParagraphState {
|
||||
isActive: boolean;
|
||||
isLoading: boolean;
|
||||
@@ -24,6 +45,11 @@ export interface ParagraphState {
|
||||
export const useParagraphMode = ({ bookKey, viewRef }: UseParagraphModeProps) => {
|
||||
const { envConfig } = useEnv();
|
||||
const { getViewSettings, setViewSettings, getProgress } = useReaderStore();
|
||||
const { getBookData } = useBookDataStore();
|
||||
|
||||
// Fixed-layout gate (D7): paragraph mode must never engage TTS sync for a
|
||||
// fixed-layout book. Mirrors how other reader code reads the flag.
|
||||
const isFixedLayout = getBookData(bookKey)?.isFixedLayout ?? false;
|
||||
|
||||
const iteratorRef = useRef<ParagraphIterator | null>(null);
|
||||
const currentDocIndexRef = useRef<number | undefined>(undefined);
|
||||
@@ -41,6 +67,29 @@ export const useParagraphMode = ({ bookKey, viewRef }: UseParagraphModeProps) =>
|
||||
paragraphCfi: string;
|
||||
docIndex: number;
|
||||
} | null>(null);
|
||||
// TTS-sync (one-way follower): TTS is the clock, paragraph mode follows it.
|
||||
const followingTtsRef = useRef(false);
|
||||
const lastSequenceSeenRef = useRef(-Infinity);
|
||||
const pendingSyncRef = useRef<{
|
||||
cfi: string;
|
||||
sequence: number;
|
||||
sectionIndex: number;
|
||||
kind?: 'word' | 'sentence';
|
||||
} | null>(null);
|
||||
// Whether the active TTS session has emitted word boundaries (Edge). Drives the
|
||||
// word-vs-sentence highlight granularity; reset when a session fully stops.
|
||||
const hasWordPositionsRef = useRef(false);
|
||||
// Holds the latest applySyncCfi so initIterator can apply a pending cross-
|
||||
// section sync after re-init without a circular useCallback dependency.
|
||||
const applySyncCfiRef = useRef<((cfi: string, highlight: boolean) => void) | null>(null);
|
||||
// Latest TTS playback-state for this book ('playing' vs not), used to derive
|
||||
// the sync status alongside the follow/pending refs.
|
||||
const ttsPlayingRef = useRef(false);
|
||||
// A TTS session exists (playing OR paused) for this book. Distinct from
|
||||
// ttsPlayingRef so a pause keeps the indicator + audio toggle "active" (the
|
||||
// mode is still on); only a full stop clears it. Drives the bar's audio toggle.
|
||||
const ttsActiveRef = useRef(false);
|
||||
const refreshTtsSyncStatusRef = useRef<(() => void) | null>(null);
|
||||
bookKeyRef.current = bookKey;
|
||||
|
||||
const [paragraphState, setParagraphState] = useState<ParagraphState>({
|
||||
@@ -51,6 +100,33 @@ export const useParagraphMode = ({ bookKey, viewRef }: UseParagraphModeProps) =>
|
||||
currentRange: null,
|
||||
});
|
||||
|
||||
const [ttsSyncStatus, setTtsSyncStatus] = useState<TtsSyncStatus>(
|
||||
isFixedLayout ? 'unsupported' : 'idle',
|
||||
);
|
||||
|
||||
// Whether a TTS session is engaged (playing or paused) for this book. Drives
|
||||
// the bar's audio toggle active/idle glyph (#3235).
|
||||
const [ttsActive, setTtsActive] = useState(false);
|
||||
|
||||
// Derive the indicator status from the current refs + the latest playback
|
||||
// state and push it to React state so the indicator re-renders. Fixed-layout
|
||||
// always wins.
|
||||
const refreshTtsSyncStatus = useCallback(() => {
|
||||
setTtsSyncStatus(() => {
|
||||
if (isFixedLayout) return 'unsupported';
|
||||
// No session at all (never started / fully stopped): idle.
|
||||
if (!ttsActiveRef.current) return 'idle';
|
||||
// Engaged but paused: keep the indicator visible (mode still on).
|
||||
if (!ttsPlayingRef.current) return 'paused';
|
||||
if (!followingTtsRef.current) return 'decoupled';
|
||||
if (pendingSyncRef.current) return 'syncing';
|
||||
return 'following';
|
||||
});
|
||||
}, [isFixedLayout]);
|
||||
// Lets initIterator refresh the status after clearing a pending sync without
|
||||
// pulling refreshTtsSyncStatus into its (widely-depended-on) deps array.
|
||||
refreshTtsSyncStatusRef.current = refreshTtsSyncStatus;
|
||||
|
||||
const paragraphConfig = getViewSettings(bookKey)?.paragraphMode ?? DEFAULT_PARAGRAPH_MODE_CONFIG;
|
||||
|
||||
const getPrimaryContent = useCallback(() => {
|
||||
@@ -135,7 +211,8 @@ export const useParagraphMode = ({ bookKey, viewRef }: UseParagraphModeProps) =>
|
||||
const resolved = view.resolveCFI(progressLocation);
|
||||
if (!resolved || resolved.index !== docIndex) return null;
|
||||
const anchor = resolved.anchor(doc);
|
||||
if (anchor instanceof Range) return anchor;
|
||||
// Realm-safe: iframe-realm Range fails top-realm `instanceof Range`.
|
||||
if (isRangeLike(anchor)) return anchor;
|
||||
if (anchor) {
|
||||
const range = doc.createRange();
|
||||
range.selectNodeContents(anchor);
|
||||
@@ -155,7 +232,8 @@ export const useParagraphMode = ({ bookKey, viewRef }: UseParagraphModeProps) =>
|
||||
const resolved = view.resolveCFI(lastParagraph.paragraphCfi);
|
||||
if (!resolved || resolved.index !== docIndex) return null;
|
||||
const anchor = resolved.anchor(doc);
|
||||
if (anchor instanceof Range) return anchor;
|
||||
// Realm-safe: iframe-realm Range fails top-realm `instanceof Range`.
|
||||
if (isRangeLike(anchor)) return anchor;
|
||||
if (anchor) {
|
||||
const range = doc.createRange();
|
||||
range.selectNodeContents(anchor);
|
||||
@@ -182,6 +260,24 @@ export const useParagraphMode = ({ bookKey, viewRef }: UseParagraphModeProps) =>
|
||||
}
|
||||
|
||||
updateStateFromIterator(false);
|
||||
|
||||
// Apply a pending cross-section TTS sync once the iterator targets the
|
||||
// CFI's section and that sync is still the latest sequence seen.
|
||||
const pending = pendingSyncRef.current;
|
||||
if (
|
||||
pending &&
|
||||
followingTtsRef.current &&
|
||||
pending.sectionIndex === currentDocIndexRef.current &&
|
||||
pending.sequence >= lastSequenceSeenRef.current
|
||||
) {
|
||||
pendingSyncRef.current = null;
|
||||
const action = decideParagraphTtsHighlight({
|
||||
kind: pending.kind,
|
||||
hasWordPositions: hasWordPositionsRef.current,
|
||||
});
|
||||
applySyncCfiRef.current?.(pending.cfi, action !== 'skip');
|
||||
refreshTtsSyncStatusRef.current?.();
|
||||
}
|
||||
return true;
|
||||
} finally {
|
||||
isProcessingRef.current = false;
|
||||
@@ -236,6 +332,113 @@ export const useParagraphMode = ({ bookKey, viewRef }: UseParagraphModeProps) =>
|
||||
});
|
||||
}, [getViewSettings, viewRef]);
|
||||
|
||||
// Sync-focus path for TTS following. Moves focus to `index` and scrolls/emits
|
||||
// exactly like focusCurrentParagraph but WITHOUT arming isFocusingRef. The
|
||||
// 200ms isFocusingRef window makes the relocate handler skip re-init; if a
|
||||
// TTS-driven focus armed it, the subsequent TTS section-change relocate would
|
||||
// be eaten and the iterator would never re-init for the new section (it would
|
||||
// then focus paragraph 0 of the wrong section). Re-entrancy from this
|
||||
// programmatic scroll's own relocate is instead prevented by the
|
||||
// lastSequenceSeen / section-match guards on the tts-position handler.
|
||||
const focusParagraphForSync = useCallback(
|
||||
async (index: number) => {
|
||||
const view = viewRef.current;
|
||||
const iterator = iteratorRef.current;
|
||||
if (!view || !iterator) return;
|
||||
|
||||
const range = iterator.goTo(index);
|
||||
if (!range) return;
|
||||
updateStateFromIterator();
|
||||
|
||||
await new Promise((r) => requestAnimationFrame(r));
|
||||
|
||||
const presentation = getParagraphPresentation(
|
||||
range.startContainer.ownerDocument,
|
||||
range,
|
||||
getViewSettings(bookKeyRef.current),
|
||||
);
|
||||
|
||||
const docIndex = currentDocIndexRef.current;
|
||||
const renderer = view.renderer as FoliateView['renderer'] & {
|
||||
goTo?: (target: { index: number; anchor: Range }) => Promise<void>;
|
||||
};
|
||||
if (docIndex !== undefined && renderer.goTo) {
|
||||
renderer.goTo({ index: docIndex, anchor: range });
|
||||
} else {
|
||||
view.renderer.scrollToAnchor?.(range);
|
||||
}
|
||||
|
||||
eventDispatcher.dispatch('paragraph-focus', {
|
||||
bookKey: bookKeyRef.current,
|
||||
range,
|
||||
index: iterator.currentIndex,
|
||||
total: iterator.length,
|
||||
presentation,
|
||||
});
|
||||
},
|
||||
[getViewSettings, viewRef, updateStateFromIterator],
|
||||
);
|
||||
|
||||
// Resolve a TTS cfi to the matching paragraph index in the current section and
|
||||
// sync-focus it. No-op when the cfi can't be resolved or maps nowhere (-1).
|
||||
// When `highlight` is set, also dispatch the spoken word/sentence offsets so
|
||||
// the overlay highlights the current text within the focused paragraph (#3235);
|
||||
// this fires even when the paragraph doesn't change (word moving within it).
|
||||
const applySyncCfi = useCallback(
|
||||
(cfi: string, highlight: boolean) => {
|
||||
const view = viewRef.current;
|
||||
const iterator = iteratorRef.current;
|
||||
const docIndex = currentDocIndexRef.current;
|
||||
if (!view || !iterator || docIndex === undefined) return;
|
||||
|
||||
let range: Range | null = null;
|
||||
try {
|
||||
const resolved = view.resolveCFI(cfi);
|
||||
if (!resolved || resolved.index !== docIndex) return;
|
||||
const content = getPrimaryContent();
|
||||
const doc = content?.doc;
|
||||
if (!doc) return;
|
||||
const anchor = resolved.anchor(doc);
|
||||
// Realm-safe: iframe-realm Range fails top-realm `instanceof Range`.
|
||||
if (isRangeLike(anchor)) {
|
||||
range = anchor;
|
||||
} else if (anchor) {
|
||||
range = doc.createRange();
|
||||
range.selectNodeContents(anchor);
|
||||
}
|
||||
} catch {
|
||||
return;
|
||||
}
|
||||
if (!range) return;
|
||||
|
||||
const index = iterator.findIndexByRange(range, iterator.currentIndex);
|
||||
if (index < 0) return;
|
||||
|
||||
// Move focus only when the spoken position crossed into another paragraph;
|
||||
// goTo() runs synchronously so iterator.current() is the target afterwards.
|
||||
if (index !== iterator.currentIndex) {
|
||||
focusParagraphForSync(index);
|
||||
}
|
||||
|
||||
if (highlight) {
|
||||
const paragraphRange = iterator.current();
|
||||
const offsets = paragraphRange
|
||||
? computeParagraphHighlightOffsets(paragraphRange, range)
|
||||
: null;
|
||||
if (offsets) {
|
||||
eventDispatcher.dispatch('paragraph-tts-highlight', {
|
||||
bookKey: bookKeyRef.current,
|
||||
index,
|
||||
start: offsets.start,
|
||||
end: offsets.end,
|
||||
});
|
||||
}
|
||||
}
|
||||
},
|
||||
[viewRef, getPrimaryContent, focusParagraphForSync],
|
||||
);
|
||||
applySyncCfiRef.current = applySyncCfi;
|
||||
|
||||
const waitForNewSection = useCallback(
|
||||
async (oldIndex: number | undefined, maxAttempts: number = 15): Promise<boolean> => {
|
||||
const view = viewRef.current;
|
||||
@@ -262,6 +465,11 @@ export const useParagraphMode = ({ bookKey, viewRef }: UseParagraphModeProps) =>
|
||||
const view = viewRef.current;
|
||||
if (!iterator || !view) return false;
|
||||
|
||||
// Manual nav decouples from TTS following until the user re-engages.
|
||||
followingTtsRef.current = false;
|
||||
pendingSyncRef.current = null;
|
||||
refreshTtsSyncStatus();
|
||||
|
||||
const range = iterator.next();
|
||||
if (range) {
|
||||
updateStateFromIterator();
|
||||
@@ -302,13 +510,25 @@ export const useParagraphMode = ({ bookKey, viewRef }: UseParagraphModeProps) =>
|
||||
focusCurrentParagraph();
|
||||
return false;
|
||||
}
|
||||
}, [viewRef, updateStateFromIterator, focusCurrentParagraph, initIterator, waitForNewSection]);
|
||||
}, [
|
||||
viewRef,
|
||||
updateStateFromIterator,
|
||||
focusCurrentParagraph,
|
||||
initIterator,
|
||||
waitForNewSection,
|
||||
refreshTtsSyncStatus,
|
||||
]);
|
||||
|
||||
const goToPrevParagraph = useCallback(async () => {
|
||||
const iterator = iteratorRef.current;
|
||||
const view = viewRef.current;
|
||||
if (!iterator || !view) return false;
|
||||
|
||||
// Manual nav decouples from TTS following until the user re-engages.
|
||||
followingTtsRef.current = false;
|
||||
pendingSyncRef.current = null;
|
||||
refreshTtsSyncStatus();
|
||||
|
||||
const range = iterator.prev();
|
||||
if (range) {
|
||||
updateStateFromIterator();
|
||||
@@ -349,7 +569,43 @@ export const useParagraphMode = ({ bookKey, viewRef }: UseParagraphModeProps) =>
|
||||
focusCurrentParagraph();
|
||||
return false;
|
||||
}
|
||||
}, [viewRef, updateStateFromIterator, focusCurrentParagraph, initIterator, waitForNewSection]);
|
||||
}, [
|
||||
viewRef,
|
||||
updateStateFromIterator,
|
||||
focusCurrentParagraph,
|
||||
initIterator,
|
||||
waitForNewSection,
|
||||
refreshTtsSyncStatus,
|
||||
]);
|
||||
|
||||
// Re-engage TTS following after a manual nav decoupled it (indicator's
|
||||
// "Resume audio" action). Sets following back on and refreshes the derived
|
||||
// status; the next tts-position event re-syncs the focused paragraph. No-op
|
||||
// on fixed-layout (sync is unsupported there).
|
||||
const reengageTtsFollow = useCallback(() => {
|
||||
if (isFixedLayout) return;
|
||||
followingTtsRef.current = true;
|
||||
refreshTtsSyncStatus();
|
||||
}, [isFixedLayout, refreshTtsSyncStatus]);
|
||||
|
||||
// Audio (TTS) toggle from the paragraph bar (#3235). When a TTS session is
|
||||
// engaged, stop it; otherwise start it from the FOCUSED paragraph with
|
||||
// start-alignment — the paragraph's range (validated live) + its section index
|
||||
// — so audio begins at the same paragraph that's highlighted. Mirrors RSVP's
|
||||
// handleToggleTtsAudio.
|
||||
const toggleTtsAudio = useCallback(() => {
|
||||
if (ttsActiveRef.current) {
|
||||
eventDispatcher.dispatch('tts-stop', { bookKey: bookKeyRef.current });
|
||||
return;
|
||||
}
|
||||
const range = iteratorRef.current?.current() ?? null;
|
||||
const docIndex = currentDocIndexRef.current;
|
||||
// The doc the focused paragraph lives in (current primary content), used to
|
||||
// validate the range is live before passing it along.
|
||||
const currentDoc = getPrimaryContent()?.doc;
|
||||
const detail = buildParagraphTtsSpeakDetail(range, docIndex, bookKeyRef.current, currentDoc);
|
||||
eventDispatcher.dispatch('tts-speak', detail);
|
||||
}, [getPrimaryContent]);
|
||||
|
||||
const goToParagraph = useCallback(
|
||||
(index: number) => {
|
||||
@@ -509,6 +765,93 @@ export const useParagraphMode = ({ bookKey, viewRef }: UseParagraphModeProps) =>
|
||||
};
|
||||
}, [toggleParagraphMode, goToNextParagraph, goToPrevParagraph, paragraphConfig.enabled]);
|
||||
|
||||
// TTS sync (one-way follower): while paragraph mode is active, follow the
|
||||
// spoken position. TTS is the clock; this never drives TTS back. Fixed-layout
|
||||
// books never engage (D7); the indicator stays 'unsupported'.
|
||||
useEffect(() => {
|
||||
if (!paragraphConfig.enabled) return;
|
||||
if (isFixedLayout) return;
|
||||
|
||||
const handlePlaybackState = (event: CustomEvent) => {
|
||||
const detail = event.detail as { bookKey?: string; state?: string } | undefined;
|
||||
if (detail?.bookKey !== bookKeyRef.current) return;
|
||||
const playing = detail.state === 'playing';
|
||||
ttsPlayingRef.current = playing;
|
||||
// A session exists while playing OR paused; only a full stop clears it.
|
||||
const active = detail.state === 'playing' || detail.state === 'paused';
|
||||
ttsActiveRef.current = active;
|
||||
setTtsActive(active);
|
||||
if (playing) {
|
||||
// Fresh engage (re-)enables following.
|
||||
followingTtsRef.current = true;
|
||||
}
|
||||
if (!active) {
|
||||
// Full stop: forget word-boundary state and clear the word highlight so a
|
||||
// later engine (which may lack word boundaries) starts clean.
|
||||
hasWordPositionsRef.current = false;
|
||||
eventDispatcher.dispatch('paragraph-tts-highlight', {
|
||||
bookKey: bookKeyRef.current,
|
||||
clear: true,
|
||||
});
|
||||
}
|
||||
refreshTtsSyncStatus();
|
||||
};
|
||||
|
||||
const handlePosition = (event: CustomEvent) => {
|
||||
const detail = event.detail as
|
||||
| {
|
||||
bookKey?: string;
|
||||
cfi?: string;
|
||||
sectionIndex?: number;
|
||||
sequence?: number;
|
||||
kind?: 'word' | 'sentence';
|
||||
}
|
||||
| undefined;
|
||||
if (detail?.bookKey !== bookKeyRef.current) return;
|
||||
if (!followingTtsRef.current) return;
|
||||
if (typeof detail.cfi !== 'string' || typeof detail.sectionIndex !== 'number') return;
|
||||
|
||||
// Drop out-of-order / stale events (dispatch awaits listeners serially and
|
||||
// callers fire-and-forget, so a slow map can land after a newer one).
|
||||
const sequence = detail.sequence ?? -Infinity;
|
||||
if (sequence <= lastSequenceSeenRef.current) return;
|
||||
lastSequenceSeenRef.current = sequence;
|
||||
|
||||
// Word vs sentence highlight granularity (Edge emits both; words win once
|
||||
// seen). Paragraph selection still runs for 'skip' so following keeps up.
|
||||
const action = decideParagraphTtsHighlight({
|
||||
kind: detail.kind,
|
||||
hasWordPositions: hasWordPositionsRef.current,
|
||||
});
|
||||
if (detail.kind === 'word') hasWordPositionsRef.current = true;
|
||||
|
||||
if (detail.sectionIndex === currentDocIndexRef.current) {
|
||||
applySyncCfi(detail.cfi, action !== 'skip');
|
||||
return;
|
||||
}
|
||||
|
||||
// Different section: don't map. Stash the latest sync and invalidate the
|
||||
// current section so the existing relocate handler re-inits the iterator
|
||||
// for the new section; the pending sync is applied once re-init completes.
|
||||
pendingSyncRef.current = {
|
||||
cfi: detail.cfi,
|
||||
sequence,
|
||||
sectionIndex: detail.sectionIndex,
|
||||
kind: detail.kind,
|
||||
};
|
||||
iteratorRef.current = null;
|
||||
refreshTtsSyncStatus();
|
||||
};
|
||||
|
||||
eventDispatcher.on('tts-playback-state', handlePlaybackState);
|
||||
eventDispatcher.on('tts-position', handlePosition);
|
||||
|
||||
return () => {
|
||||
eventDispatcher.off('tts-playback-state', handlePlaybackState);
|
||||
eventDispatcher.off('tts-position', handlePosition);
|
||||
};
|
||||
}, [paragraphConfig.enabled, isFixedLayout, applySyncCfi, refreshTtsSyncStatus]);
|
||||
|
||||
useEffect(() => {
|
||||
return () => {
|
||||
if (focusResetTimerRef.current) {
|
||||
@@ -524,11 +867,15 @@ export const useParagraphMode = ({ bookKey, viewRef }: UseParagraphModeProps) =>
|
||||
|
||||
return {
|
||||
paragraphState,
|
||||
ttsSyncStatus,
|
||||
ttsActive,
|
||||
paragraphConfig,
|
||||
toggleParagraphMode,
|
||||
goToNextParagraph,
|
||||
goToPrevParagraph,
|
||||
goToParagraph,
|
||||
toggleTtsAudio,
|
||||
reengageTtsFollow,
|
||||
focusCurrentParagraph,
|
||||
initIterator,
|
||||
};
|
||||
|
||||
@@ -66,6 +66,12 @@ export const useTTSControl = ({ bookKey, onRequestHidePanel }: UseTTSControlProp
|
||||
deinitMediaSession,
|
||||
} = useTTSMediaSession({ bookKey });
|
||||
|
||||
// Broadcast playback transitions on the app-wide bus so consumers that
|
||||
// can't read the hook-local isPlaying flag (RSVP, paragraph mode) can react.
|
||||
const emitPlaybackState = (state: 'playing' | 'paused' | 'stopped') => {
|
||||
eventDispatcher.dispatch('tts-playback-state', { bookKey, state });
|
||||
};
|
||||
|
||||
const handleTTSForward = async (event: CustomEvent) => {
|
||||
const detail = event.detail as { bookKey: string; byMark?: boolean } | undefined;
|
||||
if (detail?.bookKey !== bookKey) return;
|
||||
@@ -92,6 +98,22 @@ export const useTTSControl = ({ bookKey, onRequestHidePanel }: UseTTSControlProp
|
||||
eventDispatcher.dispatch('create-tts-highlight', { bookKey, ...sentence });
|
||||
};
|
||||
|
||||
// Set the TTS rate from the app bus. The RSVP overlay is full-screen, so its
|
||||
// rate picker can't reach the TTS panel; it dispatches `tts-set-rate` and we
|
||||
// reuse the same controller rate-change path the panel uses (handleSetRate,
|
||||
// defined below — stop→setRate→start while playing, throttled). Also persists
|
||||
// the value to viewSettings so it survives like a panel change.
|
||||
const handleTTSSetRate = (event: CustomEvent) => {
|
||||
const detail = event.detail as { bookKey: string; rate?: number } | undefined;
|
||||
if (detail?.bookKey !== bookKey || typeof detail.rate !== 'number') return;
|
||||
const viewSettings = getViewSettings(bookKey);
|
||||
if (viewSettings) {
|
||||
viewSettings.ttsRate = detail.rate;
|
||||
setViewSettings(bookKey, viewSettings);
|
||||
}
|
||||
handleSetRate(detail.rate);
|
||||
};
|
||||
|
||||
const handleTTSTogglePlay = async (event: CustomEvent) => {
|
||||
const detail = event.detail as { bookKey: string } | undefined;
|
||||
if (detail?.bookKey !== bookKey) return;
|
||||
@@ -100,10 +122,12 @@ export const useTTSControl = ({ bookKey, onRequestHidePanel }: UseTTSControlProp
|
||||
if (ttsController.state === 'playing') {
|
||||
setIsPlaying(false);
|
||||
setIsPaused(true);
|
||||
emitPlaybackState('paused');
|
||||
await ttsController.pause();
|
||||
} else {
|
||||
setIsPlaying(true);
|
||||
setIsPaused(false);
|
||||
emitPlaybackState('playing');
|
||||
if (ttsController.state === 'paused') {
|
||||
await ttsController.resume();
|
||||
} else {
|
||||
@@ -118,6 +142,7 @@ export const useTTSControl = ({ bookKey, onRequestHidePanel }: UseTTSControlProp
|
||||
eventDispatcher.on('tts-forward', handleTTSForward);
|
||||
eventDispatcher.on('tts-backward', handleTTSBackward);
|
||||
eventDispatcher.on('tts-toggle-play', handleTTSTogglePlay);
|
||||
eventDispatcher.on('tts-set-rate', handleTTSSetRate);
|
||||
eventDispatcher.on('tts-highlight-sentence', handleTTSHighlightSentence);
|
||||
return () => {
|
||||
eventDispatcher.off('tts-speak', handleTTSSpeak);
|
||||
@@ -125,6 +150,7 @@ export const useTTSControl = ({ bookKey, onRequestHidePanel }: UseTTSControlProp
|
||||
eventDispatcher.off('tts-forward', handleTTSForward);
|
||||
eventDispatcher.off('tts-backward', handleTTSBackward);
|
||||
eventDispatcher.off('tts-toggle-play', handleTTSTogglePlay);
|
||||
eventDispatcher.off('tts-set-rate', handleTTSSetRate);
|
||||
eventDispatcher.off('tts-highlight-sentence', handleTTSHighlightSentence);
|
||||
if (ttsControllerRef.current) {
|
||||
ttsControllerRef.current.shutdown();
|
||||
@@ -293,15 +319,31 @@ export const useTTSControl = ({ bookKey, onRequestHidePanel }: UseTTSControlProp
|
||||
}
|
||||
};
|
||||
|
||||
// Republish the controller's canonical position signal onto the app-wide
|
||||
// bus so paragraph mode + RSVP can follow TTS without touching the
|
||||
// controller. This MUST be its own listener: handleHighlightMark /
|
||||
// handleHighlightWord early-return on following-suppression and text
|
||||
// selection, which would silently stop the modes from following. The
|
||||
// forward fires on every controller 'tts-position', gated only by the
|
||||
// listener's lifecycle (it exists only while the controller does).
|
||||
const handlePosition = (e: Event) => {
|
||||
eventDispatcher.dispatch('tts-position', {
|
||||
bookKey,
|
||||
...(e as CustomEvent).detail,
|
||||
});
|
||||
};
|
||||
|
||||
ttsController.addEventListener('tts-need-auth', handleNeedAuth);
|
||||
ttsController.addEventListener('tts-speak-mark', handleSpeakMark);
|
||||
ttsController.addEventListener('tts-highlight-mark', handleHighlightMark);
|
||||
ttsController.addEventListener('tts-highlight-word', handleHighlightWord);
|
||||
ttsController.addEventListener('tts-position', handlePosition);
|
||||
return () => {
|
||||
ttsController.removeEventListener('tts-need-auth', handleNeedAuth);
|
||||
ttsController.removeEventListener('tts-speak-mark', handleSpeakMark);
|
||||
ttsController.removeEventListener('tts-highlight-mark', handleHighlightMark);
|
||||
ttsController.removeEventListener('tts-highlight-word', handleHighlightWord);
|
||||
ttsController.removeEventListener('tts-position', handlePosition);
|
||||
};
|
||||
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||
}, [ttsController, bookKey]);
|
||||
@@ -473,6 +515,7 @@ export const useTTSControl = ({ bookKey, onRequestHidePanel }: UseTTSControlProp
|
||||
setTtsController(null);
|
||||
getView(bookKey)?.deselect();
|
||||
setIsPlaying(false);
|
||||
emitPlaybackState('stopped');
|
||||
onRequestHidePanel?.();
|
||||
setShowIndicator(false);
|
||||
setShowBackToCurrentTTSLocation(false);
|
||||
@@ -583,6 +626,7 @@ export const useTTSControl = ({ bookKey, onRequestHidePanel }: UseTTSControlProp
|
||||
if (ssml) {
|
||||
const lang = parseSSMLLang(ssml, primaryLang) || 'en';
|
||||
setIsPlaying(true);
|
||||
emitPlaybackState('playing');
|
||||
setTtsLang(lang);
|
||||
|
||||
ttsController.setLang(lang);
|
||||
@@ -619,10 +663,12 @@ export const useTTSControl = ({ bookKey, onRequestHidePanel }: UseTTSControlProp
|
||||
if (isPlaying) {
|
||||
setIsPlaying(false);
|
||||
setIsPaused(true);
|
||||
emitPlaybackState('paused');
|
||||
await ttsController.pause();
|
||||
} else if (isPaused) {
|
||||
setIsPlaying(true);
|
||||
setIsPaused(false);
|
||||
emitPlaybackState('playing');
|
||||
// start for forward/backward/setvoice-paused
|
||||
// set rate don't pause the tts
|
||||
if (ttsController.state === 'paused') {
|
||||
@@ -661,8 +707,10 @@ export const useTTSControl = ({ bookKey, onRequestHidePanel }: UseTTSControlProp
|
||||
if (ttsController) {
|
||||
setIsPlaying(false);
|
||||
setIsPaused(true);
|
||||
emitPlaybackState('paused');
|
||||
await ttsController.pause();
|
||||
}
|
||||
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||
}, []);
|
||||
|
||||
// Rate/voice/timeout/bar controls
|
||||
|
||||
Reference in New Issue
Block a user