fix(tts): keep native System TTS reading past unspeakable chunks offline (#4613, #4408) (#4716)

Android System TTS (and iOS) read-aloud could stop offline and refuse to
continue — #4613 "stops at the end of the chapter, won't advance" and #4408
"stops at random intervals" — after which the play/headphone controls felt
wedged.

Root cause: `TTSController.#speak` only auto-advances when the last event code
is `end`. The native client surfaces an offline engine failure as a terminal
`error` code (Android `UtteranceProgressListener.onError`). This typically
happens on a specific utterance the offline engine can't synthesize — e.g. an
unsupported character — characteristically the first utterance after a chapter
boundary, even with a local/offline voice (online, engines often fall back to
network synthesis, which is why it only breaks offline). On `error` the
controller never called `forward()` and left `state` stuck at `playing`, so
playback dead-ended and the controls couldn't recover. Edge/Web clients throw
instead (handled by `error()`), so only the native client hit this.

Fix (native-scoped, no change to the Edge/Web path): when the active client is
the native client and an utterance ends with a terminal `error` (still playing,
not aborted, not one-time), skip that chunk and advance just as a normal `end`
would — re-speaking the same unsynthesizable text would only fail again. A
consecutive-error cap stops playback gracefully if the engine can't speak
anything, so a wholly-unusable engine doesn't silently race to the end of the
book and the state machine always leaves `playing`.

Tests: tts-controller covers skip-on-error advancing past a bad chunk, and the
consecutive-error cap stopping gracefully (bounded, not wedged in playing).

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Huang Xin
2026-06-22 10:50:37 +08:00
committed by GitHub
parent 9155ae627c
commit b87c735c1e
2 changed files with 130 additions and 0 deletions
@@ -1116,6 +1116,92 @@ describe('TTSController', () => {
});
});
// Regression: Android System TTS (and iOS) read offline can report a terminal
// 'error' for an utterance it cannot synthesize — typically a specific
// unsupported character, hit characteristically on the first utterance after
// a chapter boundary even with a local/offline voice (online the engine often
// network-falls-back, which is why it only breaks offline). #speak only
// auto-advances on 'end', so without handling, one such error dead-ends
// playback ("stops at the end of the chapter") with the controls wedged in
// 'playing'. Re-speaking the same text would just fail again, so the
// controller skips the bad chunk and advances, bounding consecutive failures
// so a wholly-unusable engine still stops gracefully. See #4613, #4408.
describe('native TTS offline error recovery (#4613, #4408)', () => {
// An Android controller whose ACTIVE client is the native client, so the
// native-scoped recovery in #speak() is exercised.
const makeAndroidNativeController = async () => {
const androidService = createMockAppService(true);
const c = new TTSController(androidService, mockView);
await c.init();
c.ttsClient = c.ttsNativeClient!;
await c.initViewTTS(0);
return c;
};
// A native speak() mock that always reports a terminal 'error' for real
// (non-preload) utterances — i.e. a deterministically unspeakable chunk.
// Preload calls (used to warm caches) resolve immediately like the real
// client and never count as attempts.
const alwaysErrorSpeakMock = (state: { attempts: number }) =>
async function* (
_ssml: string,
_signal: AbortSignal,
preload?: boolean,
): AsyncGenerator<TTSMessageEvent> {
if (preload) {
yield { code: 'end' };
return;
}
state.attempts += 1;
yield { code: 'error', message: 'TTS playback error:-8' };
};
test('skips a chunk the engine cannot speak and advances instead of dead-ending', async () => {
const c = await makeAndroidNativeController();
// Stub forward() so the skip is observable without recursing through the
// mock sections; the point is that an error triggers an advance at all
// (retrying the same unspeakable text would be futile).
const forwardSpy = vi.spyOn(c, 'forward').mockResolvedValue();
const state = { attempts: 0 };
vi.mocked(c.ttsNativeClient!.speak).mockImplementation(alwaysErrorSpeakMock(state));
c.speak('<speak>bad-char</speak>');
await vi.waitFor(
() => {
expect(forwardSpy).toHaveBeenCalled(); // advanced past the bad chunk
},
{ timeout: 5000 },
);
// It advanced rather than freezing in a phantom 'playing' halt.
expect(c.state).toBe('playing');
});
test('stops gracefully after a run of consecutive unspeakable chunks', async () => {
const c = await makeAndroidNativeController();
// Real forward(): each error skips to the next (mock) chunk, which also
// errors, until the consecutive-error cap stops playback.
const state = { attempts: 0 };
vi.mocked(c.ttsNativeClient!.speak).mockImplementation(alwaysErrorSpeakMock(state));
c.speak('<speak>bad-char</speak>');
// It skips past each unspeakable chunk — attempts climb past 1 (not an
// immediate halt) — until the consecutive-error cap is reached. (We key
// off attempts, not state: the controller starts 'stopped' and forward()
// transiently re-enters 'stopped' between chunks.)
await vi.waitFor(() => expect(state.attempts).toBeGreaterThanOrEqual(5), { timeout: 8000 });
// Let the cap-stop settle, then confirm it terminated (bounded, not
// racing to the end of the book) and is no longer playing.
await new Promise((r) => setTimeout(r, 150));
expect(c.state).not.toBe('playing');
expect(state.attempts).toBeLessThanOrEqual(10);
});
});
describe('preloadSSML', () => {
test('does nothing when ssml is undefined', async () => {
await controller.preloadSSML(undefined, new AbortController().signal);
@@ -25,6 +25,18 @@ import {
// across sessions.
let ttsPositionSequence = 0;
// Native TTS (Android System TTS / iOS) can report a terminal 'error' for an
// utterance it cannot synthesize offline — typically a specific unsupported
// character, hit characteristically on the first utterance after a chapter
// boundary even with a local/offline voice (online the engine often
// network-falls-back, which is why it only breaks offline). #speak only
// auto-advances on 'end', so without handling, a single such error dead-ends
// playback and wedges the controls in 'playing'. Re-speaking the same text
// would just fail again, so we skip the bad chunk and advance — bounding
// consecutive failures so a wholly-unusable engine still stops gracefully
// instead of silently racing to the end of the book. See #4613, #4408.
const TTS_NATIVE_SPEAK_MAX_CONSECUTIVE_ERRORS = 5;
type TTSState =
| 'stopped'
| 'playing'
@@ -44,6 +56,10 @@ export class TTSController extends EventTarget {
preprocessCallback?: (ssml: string) => Promise<string>;
onSectionChange?: (sectionIndex: number) => Promise<void>;
#nossmlCnt: number = 0;
// Consecutive native-TTS utterances that ended in a terminal 'error' without
// a successful 'end' in between. Reset on success; caps skip-on-error so a
// wholly-unusable engine stops instead of racing to the book end. See #4613.
#consecutiveSpeakErrors: number = 0;
#currentSpeakAbortController: AbortController | null = null;
#currentSpeakPromise: Promise<void> | null = null;
@@ -400,6 +416,9 @@ export class TTSController extends EventTarget {
}
await this.preloadSSML(ssml, signal);
}
// Only the native client surfaces an offline engine failure as a
// terminal 'error' code (Edge/Web throw, which the catch below handles).
const canSkipOnError = this.ttsClient === this.ttsNativeClient;
const iter = await this.ttsClient.speak(ssml, signal);
let lastCode;
for await (const { code } of iter) {
@@ -411,8 +430,33 @@ export class TTSController extends EventTarget {
}
if (lastCode === 'end' && this.state === 'playing' && !oneTime) {
this.#consecutiveSpeakErrors = 0;
resolve();
await this.forward();
} else if (
lastCode === 'error' &&
canSkipOnError &&
!signal.aborted &&
this.state === 'playing' &&
!oneTime
) {
// The native engine reported it can't speak this chunk. Offline this
// is almost always a specific unsynthesizable utterance (e.g. an
// unsupported character) that would fail every time, not a transient
// glitch — so retrying the same text is futile. Skip it and advance
// exactly as a normal 'end' would, so one bad chunk (often the first
// utterance across a chapter boundary) can't strand playback with the
// controls wedged in 'playing'. Bound consecutive failures so a
// wholly-unusable engine stops gracefully instead of silently racing
// to the end of the book. See #4613, #4408.
this.#consecutiveSpeakErrors++;
resolve();
if (this.#consecutiveSpeakErrors <= TTS_NATIVE_SPEAK_MAX_CONSECUTIVE_ERRORS) {
await this.forward();
} else {
this.#consecutiveSpeakErrors = 0;
await this.stop();
}
}
resolve();
} catch (e) {