Highlight each word as it is spoken (Edge TTS only) instead of keeping the
whole sentence highlighted, and keep the view tracking the spoken word
across page boundaries.
Word boundaries
- Capture Edge's audio.metadata WordBoundary frames (offset/duration in
100ns ticks plus the verbatim input-text span) in the Tauri, browser, and
Cloudflare-Workers WebSocket transports.
- Carry boundaries through the authenticated HTTPS proxy route via an
X-TTS-Word-Boundaries response header (percent-encoded JSON, ASCII-safe),
so word highlighting works on the web where the browser cannot open the
wss connection directly. Cache them alongside the audio blob URL.
Highlighting
- Sync a requestAnimationFrame loop to audio.currentTime against the
boundary table and highlight the word sub-range within the spoken
sentence. Synthesis stays sentence-level (natural prosody); only the
visual highlight is word-level.
- Suppress the sentence highlight when the active client reports word
boundaries and draw the first word immediately, so the whole sentence
never flashes before the first word. Fall back to the sentence highlight
when a chunk has no boundaries (other engines, empty metadata).
- Re-apply the current word (not the sentence) when the view relocates.
Page following
- Turn the page as soon as the spoken word crosses a page boundary (a
tts-highlight-word event scrolls only when the word is outside the visible
range), instead of waiting for the next sentence.
- Check the word's position for the "back to TTS location" badge so it no
longer appears while the view follows the word onto the next page.
Also fixes a pre-existing bug where the browser WebSocket was constructed
with an options object (valid only for the Node ws package), which threw in
browsers and made the wss path unusable on the web.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Huang Xin
a56cc6c61a