Android System TTS (and iOS) read-aloud could stop offline and refuse to continue — #4613 "stops at the end of the chapter, won't advance" and #4408 "stops at random intervals" — after which the play/headphone controls felt wedged. Root cause: `TTSController.#speak` only auto-advances when the last event code is `end`. The native client surfaces an offline engine failure as a terminal `error` code (Android `UtteranceProgressListener.onError`). This typically happens on a specific utterance the offline engine can't synthesize — e.g. an unsupported character — characteristically the first utterance after a chapter boundary, even with a local/offline voice (online, engines often fall back to network synthesis, which is why it only breaks offline). On `error` the controller never called `forward()` and left `state` stuck at `playing`, so playback dead-ended and the controls couldn't recover. Edge/Web clients throw instead (handled by `error()`), so only the native client hit this. Fix (native-scoped, no change to the Edge/Web path): when the active client is the native client and an utterance ends with a terminal `error` (still playing, not aborted, not one-time), skip that chunk and advance just as a normal `end` would — re-speaking the same unsynthesizable text would only fail again. A consecutive-error cap stops playback gracefully if the engine can't speak anything, so a wholly-unusable engine doesn't silently race to the end of the book and the state machine always leaves `playing`. Tests: tts-controller covers skip-on-error advancing past a bad chunk, and the consecutive-error cap stopping gracefully (bounded, not wedged in playing). Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -1116,6 +1116,92 @@ describe('TTSController', () => {
|
||||
});
|
||||
});
|
||||
|
||||
// Regression: Android System TTS (and iOS) read offline can report a terminal
|
||||
// 'error' for an utterance it cannot synthesize — typically a specific
|
||||
// unsupported character, hit characteristically on the first utterance after
|
||||
// a chapter boundary even with a local/offline voice (online the engine often
|
||||
// network-falls-back, which is why it only breaks offline). #speak only
|
||||
// auto-advances on 'end', so without handling, one such error dead-ends
|
||||
// playback ("stops at the end of the chapter") with the controls wedged in
|
||||
// 'playing'. Re-speaking the same text would just fail again, so the
|
||||
// controller skips the bad chunk and advances, bounding consecutive failures
|
||||
// so a wholly-unusable engine still stops gracefully. See #4613, #4408.
|
||||
describe('native TTS offline error recovery (#4613, #4408)', () => {
|
||||
// An Android controller whose ACTIVE client is the native client, so the
|
||||
// native-scoped recovery in #speak() is exercised.
|
||||
const makeAndroidNativeController = async () => {
|
||||
const androidService = createMockAppService(true);
|
||||
const c = new TTSController(androidService, mockView);
|
||||
await c.init();
|
||||
c.ttsClient = c.ttsNativeClient!;
|
||||
await c.initViewTTS(0);
|
||||
return c;
|
||||
};
|
||||
|
||||
// A native speak() mock that always reports a terminal 'error' for real
|
||||
// (non-preload) utterances — i.e. a deterministically unspeakable chunk.
|
||||
// Preload calls (used to warm caches) resolve immediately like the real
|
||||
// client and never count as attempts.
|
||||
const alwaysErrorSpeakMock = (state: { attempts: number }) =>
|
||||
async function* (
|
||||
_ssml: string,
|
||||
_signal: AbortSignal,
|
||||
preload?: boolean,
|
||||
): AsyncGenerator<TTSMessageEvent> {
|
||||
if (preload) {
|
||||
yield { code: 'end' };
|
||||
return;
|
||||
}
|
||||
state.attempts += 1;
|
||||
yield { code: 'error', message: 'TTS playback error:-8' };
|
||||
};
|
||||
|
||||
test('skips a chunk the engine cannot speak and advances instead of dead-ending', async () => {
|
||||
const c = await makeAndroidNativeController();
|
||||
// Stub forward() so the skip is observable without recursing through the
|
||||
// mock sections; the point is that an error triggers an advance at all
|
||||
// (retrying the same unspeakable text would be futile).
|
||||
const forwardSpy = vi.spyOn(c, 'forward').mockResolvedValue();
|
||||
|
||||
const state = { attempts: 0 };
|
||||
vi.mocked(c.ttsNativeClient!.speak).mockImplementation(alwaysErrorSpeakMock(state));
|
||||
|
||||
c.speak('<speak>bad-char</speak>');
|
||||
|
||||
await vi.waitFor(
|
||||
() => {
|
||||
expect(forwardSpy).toHaveBeenCalled(); // advanced past the bad chunk
|
||||
},
|
||||
{ timeout: 5000 },
|
||||
);
|
||||
// It advanced rather than freezing in a phantom 'playing' halt.
|
||||
expect(c.state).toBe('playing');
|
||||
});
|
||||
|
||||
test('stops gracefully after a run of consecutive unspeakable chunks', async () => {
|
||||
const c = await makeAndroidNativeController();
|
||||
// Real forward(): each error skips to the next (mock) chunk, which also
|
||||
// errors, until the consecutive-error cap stops playback.
|
||||
|
||||
const state = { attempts: 0 };
|
||||
vi.mocked(c.ttsNativeClient!.speak).mockImplementation(alwaysErrorSpeakMock(state));
|
||||
|
||||
c.speak('<speak>bad-char</speak>');
|
||||
|
||||
// It skips past each unspeakable chunk — attempts climb past 1 (not an
|
||||
// immediate halt) — until the consecutive-error cap is reached. (We key
|
||||
// off attempts, not state: the controller starts 'stopped' and forward()
|
||||
// transiently re-enters 'stopped' between chunks.)
|
||||
await vi.waitFor(() => expect(state.attempts).toBeGreaterThanOrEqual(5), { timeout: 8000 });
|
||||
|
||||
// Let the cap-stop settle, then confirm it terminated (bounded, not
|
||||
// racing to the end of the book) and is no longer playing.
|
||||
await new Promise((r) => setTimeout(r, 150));
|
||||
expect(c.state).not.toBe('playing');
|
||||
expect(state.attempts).toBeLessThanOrEqual(10);
|
||||
});
|
||||
});
|
||||
|
||||
describe('preloadSSML', () => {
|
||||
test('does nothing when ssml is undefined', async () => {
|
||||
await controller.preloadSSML(undefined, new AbortController().signal);
|
||||
|
||||
@@ -25,6 +25,18 @@ import {
|
||||
// across sessions.
|
||||
let ttsPositionSequence = 0;
|
||||
|
||||
// Native TTS (Android System TTS / iOS) can report a terminal 'error' for an
|
||||
// utterance it cannot synthesize offline — typically a specific unsupported
|
||||
// character, hit characteristically on the first utterance after a chapter
|
||||
// boundary even with a local/offline voice (online the engine often
|
||||
// network-falls-back, which is why it only breaks offline). #speak only
|
||||
// auto-advances on 'end', so without handling, a single such error dead-ends
|
||||
// playback and wedges the controls in 'playing'. Re-speaking the same text
|
||||
// would just fail again, so we skip the bad chunk and advance — bounding
|
||||
// consecutive failures so a wholly-unusable engine still stops gracefully
|
||||
// instead of silently racing to the end of the book. See #4613, #4408.
|
||||
const TTS_NATIVE_SPEAK_MAX_CONSECUTIVE_ERRORS = 5;
|
||||
|
||||
type TTSState =
|
||||
| 'stopped'
|
||||
| 'playing'
|
||||
@@ -44,6 +56,10 @@ export class TTSController extends EventTarget {
|
||||
preprocessCallback?: (ssml: string) => Promise<string>;
|
||||
onSectionChange?: (sectionIndex: number) => Promise<void>;
|
||||
#nossmlCnt: number = 0;
|
||||
// Consecutive native-TTS utterances that ended in a terminal 'error' without
|
||||
// a successful 'end' in between. Reset on success; caps skip-on-error so a
|
||||
// wholly-unusable engine stops instead of racing to the book end. See #4613.
|
||||
#consecutiveSpeakErrors: number = 0;
|
||||
#currentSpeakAbortController: AbortController | null = null;
|
||||
#currentSpeakPromise: Promise<void> | null = null;
|
||||
|
||||
@@ -400,6 +416,9 @@ export class TTSController extends EventTarget {
|
||||
}
|
||||
await this.preloadSSML(ssml, signal);
|
||||
}
|
||||
// Only the native client surfaces an offline engine failure as a
|
||||
// terminal 'error' code (Edge/Web throw, which the catch below handles).
|
||||
const canSkipOnError = this.ttsClient === this.ttsNativeClient;
|
||||
const iter = await this.ttsClient.speak(ssml, signal);
|
||||
let lastCode;
|
||||
for await (const { code } of iter) {
|
||||
@@ -411,8 +430,33 @@ export class TTSController extends EventTarget {
|
||||
}
|
||||
|
||||
if (lastCode === 'end' && this.state === 'playing' && !oneTime) {
|
||||
this.#consecutiveSpeakErrors = 0;
|
||||
resolve();
|
||||
await this.forward();
|
||||
} else if (
|
||||
lastCode === 'error' &&
|
||||
canSkipOnError &&
|
||||
!signal.aborted &&
|
||||
this.state === 'playing' &&
|
||||
!oneTime
|
||||
) {
|
||||
// The native engine reported it can't speak this chunk. Offline this
|
||||
// is almost always a specific unsynthesizable utterance (e.g. an
|
||||
// unsupported character) that would fail every time, not a transient
|
||||
// glitch — so retrying the same text is futile. Skip it and advance
|
||||
// exactly as a normal 'end' would, so one bad chunk (often the first
|
||||
// utterance across a chapter boundary) can't strand playback with the
|
||||
// controls wedged in 'playing'. Bound consecutive failures so a
|
||||
// wholly-unusable engine stops gracefully instead of silently racing
|
||||
// to the end of the book. See #4613, #4408.
|
||||
this.#consecutiveSpeakErrors++;
|
||||
resolve();
|
||||
if (this.#consecutiveSpeakErrors <= TTS_NATIVE_SPEAK_MAX_CONSECUTIVE_ERRORS) {
|
||||
await this.forward();
|
||||
} else {
|
||||
this.#consecutiveSpeakErrors = 0;
|
||||
await this.stop();
|
||||
}
|
||||
}
|
||||
resolve();
|
||||
} catch (e) {
|
||||
|
||||
Reference in New Issue
Block a user