fix(android): background TTS media controls + lock-screen scrubber/seek + Edge click fix (#4994)

* fix(android): keep background TTS media controls alive when backgrounded The media-session update commands pushed metadata and playback state to MediaPlaybackService via Context.startService(), which Android 8+ rejects with "app is in background" once the app leaves the foreground. Every per-sentence update then failed, so the lock-screen control went stale and the foreground service lost the notification refresh that keeps it alive, dropping background playback. Deliver updates to the running service in-process (the pattern the existing requestDeactivation() already uses) so they can never trigger a service start. Also request POST_NOTIFICATIONS whenever the session activates, not only when alwaysInForeground is set, so the foreground-service media notification (the lock-screen control) is not suppressed on Android 13+. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(android): show section duration and enable seek on the TTS media session The Edge/WebAudio TTS engine already reports an estimated section timeline (position and duration) to the media session on every sentence, but the Android service never surfaced it: the lock screen had no scrubber and no seek. Set METADATA_KEY_DURATION so the scrubber shows its length, add ACTION_SEEK_TO, and handle onSeekTo by handing the target back to the JS controller (seekToTime) through the existing media-session-seek event. Playback-state updates that only flip play/pause omit position and duration, so the service now preserves the last known values instead of resetting the scrubber to 0. Native TextToSpeech has no timeline (duration stays 0), so the scrubber simply does not appear there. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(android): harden TTS foreground-service promotion + add diagnostics On MIUI the media service was reclaimed on idle ("Stopping service due to app idle"), which means startForeground never promoted it to a real foreground service. Promote with ServiceCompat and an explicit FOREGROUND_SERVICE_TYPE_MEDIA_PLAYBACK type (robust on targetSdk 34+) and log any failure instead of swallowing it. Decouple the POST_NOTIFICATIONS request from the service start in setActive: a throwing or hung permission request no longer aborts set_media_session_active, so the foreground service always starts. Add trace logging (set_media_session_active, activateSession, startForeground) to pinpoint the promotion path on-device. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(android): drop required keepAppInForeground so the TTS service starts The set_media_session_active payload required keep_app_in_foreground, but the media bridge activates with just { active: true }. Tauri rejected the invoke at the serde layer ("missing field keepAppInForeground") before the command ran, so the foreground service never started: no media notification, and Android 15 audio hardening then muted background playback. The flag was dead everywhere: no platform read it (the foreground service always starts and the POST_NOTIFICATIONS request is now unconditional). Remove it from the Rust, Kotlin, iOS, and TS payloads. Effective behavior is always-foreground. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(tts): fade sentence-buffer edges to stop Edge TTS clicks The silence trim (findSpeechBounds) cuts at an amplitude threshold, not a zero crossing, so each Edge TTS sentence buffer begins and ends on a non-zero sample. An AudioBufferSourceNode steps straight from/to silence at its edges, and that discontinuity clicks/pops between sentences (WSOLA is not involved: it is a no-op at rate 1.0 and cross-fades its internal splices). Apply a ~3ms linear fade to each buffer's own copy after WSOLA so the edges ramp to zero. The trim and the controlled inter-sentence gap are unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(android): remove the dead "Background Read Aloud" setting alwaysInForeground only drove the removed keepAppInForeground flag; nothing reads it now (the foreground service always starts, and POST_NOTIFICATIONS is requested at play time). Remove the setting, its default, the Android library-menu toggle (which just requested the notification permission), and prune the now-unused i18n key from all locales. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-07 20:35:53 +09:00
parent 3ce5a5c8e3
commit ab628fe258
47 changed files with 290 additions and 155 deletions
@@ -7,6 +7,7 @@ import android.app.NotificationManager
 import android.app.PendingIntent
 import android.content.Context
 import android.content.Intent
+import android.content.pm.ServiceInfo
 import android.os.Build
 import android.os.Bundle
 import android.os.Handler
@@ -29,14 +30,12 @@ import androidx.media3.common.MediaItem
 import androidx.media3.common.Player
 import androidx.media3.exoplayer.ExoPlayer
 import app.tauri.plugin.JSObject
-import kotlinx.coroutines.*

 class MediaPlaybackService : MediaBrowserServiceCompat() {
    private var mediaSession: MediaSessionCompat? = null
    private lateinit var player: ExoPlayer
    private lateinit var stateBuilder: PlaybackStateCompat.Builder
    private lateinit var audioManager: AudioManager
-    private val serviceScope = CoroutineScope(SupervisorJob() + Dispatchers.Main)

    // True only between session activation (TTS playback started) and
    // deactivation. Android Auto can bind this service at any time to browse,
@@ -73,6 +72,15 @@ class MediaPlaybackService : MediaBrowserServiceCompat() {
        var currentArtist: String = "Reading your content"
        var currentArtwork: Bitmap? = null

+        // Estimated section timeline (Edge/WebAudio engine only) in milliseconds.
+        // Drives the lock-screen scrubber: position is the thumb, duration is
+        // the track length. Native TextToSpeech has no timeline and leaves
+        // duration at 0, so the scrubber simply does not appear there.
+        @Volatile
+        var currentPositionMs: Long = 0L
+        @Volatile
+        var currentDurationMs: Long = 0L
+
        @Volatile
        private var instance: MediaPlaybackService? = null

@@ -85,6 +93,33 @@ class MediaPlaybackService : MediaBrowserServiceCompat() {
            val service = instance ?: return
            Handler(Looper.getMainLooper()).post { service.deactivateSession() }
        }
+
+        // Deliver metadata/state updates to the live service in-process rather
+        // than via startService(): once the app is backgrounded, startService()
+        // is rejected with "app is in background" (the Android 8+ background
+        // service-start restriction), which silently dropped every playback
+        // update — killing the lock-screen control and the notification refresh
+        // that keeps the foreground service alive. A direct call on the running
+        // instance is not a service *start*, so it is exempt. The statics are
+        // refreshed regardless so a not-yet-created service picks them up when
+        // it activates.
+        fun pushMetadata(title: String, artist: String, artwork: Bitmap?) {
+            currentTitle = title
+            currentArtist = artist
+            if (artwork != null) currentArtwork = artwork
+            val service = instance ?: return
+            Handler(Looper.getMainLooper()).post { service.applyMetadata() }
+        }
+
+        // position/duration are null when the update only reports a play/pause
+        // flip (that payload omits them); keep the last known values so the
+        // scrubber does not snap back to 0 on pause.
+        fun pushPlaybackState(playing: Boolean, position: Long?, duration: Long?) {
+            if (position != null) currentPositionMs = position
+            if (duration != null) currentDurationMs = duration
+            val service = instance ?: return
+            Handler(Looper.getMainLooper()).post { service.applyPlaybackState(playing) }
+        }
    }

    override fun onCreate() {
@@ -102,6 +137,7 @@ class MediaPlaybackService : MediaBrowserServiceCompat() {
                PlaybackStateCompat.ACTION_STOP or
                PlaybackStateCompat.ACTION_SKIP_TO_NEXT or
                PlaybackStateCompat.ACTION_SKIP_TO_PREVIOUS or
+                PlaybackStateCompat.ACTION_SEEK_TO or
                PlaybackStateCompat.ACTION_PLAY_FROM_MEDIA_ID or
                PlaybackStateCompat.ACTION_PLAY_FROM_SEARCH
            )
@@ -121,6 +157,7 @@ class MediaPlaybackService : MediaBrowserServiceCompat() {
    }

    private fun activateSession() {
+        Log.d("MediaPlaybackService", "activateSession (wasActive=$sessionActive)")
        if (!sessionActive) {
            sessionActive = true

@@ -193,6 +230,18 @@ class MediaPlaybackService : MediaBrowserServiceCompat() {
            pluginEventTrigger?.invoke("media-session-previous", JSObject())
        }

+        // Scrubber drag: hand the target back to the JS controller, which owns
+        // the real audio timeline (seekToTime), and optimistically move the
+        // thumb so the lock screen feels responsive before the seek lands.
+        override fun onSeekTo(pos: Long) {
+            currentPositionMs = pos
+            pluginEventTrigger?.invoke("media-session-seek", JSObject().apply { put("position", pos) })
+            val state = if (player.isPlaying) PlaybackStateCompat.STATE_PLAYING else PlaybackStateCompat.STATE_PAUSED
+            mediaSession?.setPlaybackState(
+                stateBuilder.setState(state, pos, 1f).build()
+            )
+        }
+
        override fun onPlayFromMediaId(mediaId: String?, extras: Bundle?) {
            onPlay()
        }
@@ -211,12 +260,78 @@ class MediaPlaybackService : MediaBrowserServiceCompat() {
        showNotification(state)
    }

+    // Last section duration written to the session metadata; the scrubber only
+    // needs a metadata refresh when it actually changes (per section), not on
+    // every position tick.
+    private var appliedDurationMs: Long = -1L
+
+    private fun buildMediaMetadata(): MediaMetadataCompat {
+        return MediaMetadataCompat.Builder()
+            .putString(MediaMetadataCompat.METADATA_KEY_TITLE, currentTitle)
+            .putString(MediaMetadataCompat.METADATA_KEY_ARTIST, currentArtist)
+            .putBitmap(MediaMetadataCompat.METADATA_KEY_ALBUM_ART, currentArtwork)
+            .putLong(MediaMetadataCompat.METADATA_KEY_DURATION, currentDurationMs)
+            .build()
+    }
+
+    // Push the current statics into the live session + notification. Invoked
+    // in-process from Companion.pushMetadata on the main thread.
+    private fun applyMetadata() {
+        appliedDurationMs = currentDurationMs
+        mediaSession?.setMetadata(buildMediaMetadata())
+        notifyChildrenChanged(MEDIA_ROOT_ID)
+        if (sessionActive) {
+            showNotification(
+                if (player.isPlaying) PlaybackStateCompat.STATE_PLAYING
+                else PlaybackStateCompat.STATE_PAUSED
+            )
+        }
+    }
+
+    // Reflect the WebView/TextToSpeech playback state onto the silent
+    // keep-alive player, the media session, and the notification. Invoked
+    // in-process from Companion.pushPlaybackState on the main thread; reads the
+    // preserved position/duration statics.
+    private fun applyPlaybackState(playing: Boolean) {
+        if (!sessionActive) return
+        if (playing && !player.isPlaying) {
+            player.play()
+        } else if (!playing && player.isPlaying) {
+            player.pause()
+        }
+        player.seekTo(currentPositionMs)
+        val state = if (playing) PlaybackStateCompat.STATE_PLAYING else PlaybackStateCompat.STATE_PAUSED
+        mediaSession?.setPlaybackState(
+            stateBuilder.setState(state, currentPositionMs, 1f).build()
+        )
+        // Refresh the scrubber length when the section duration changes.
+        if (currentDurationMs != appliedDurationMs) {
+            appliedDurationMs = currentDurationMs
+            mediaSession?.setMetadata(buildMediaMetadata())
+        }
+        showNotification(state)
+    }
+
    private fun showNotification(playbackState: Int) {
        if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
            val channel = NotificationChannel(CHANNEL_ID, "Media Controls", NotificationManager.IMPORTANCE_LOW)
            getSystemService(NotificationManager::class.java).createNotificationChannel(channel)
        }
-        startForeground(NOTIFICATION_ID, buildNotification(playbackState))
+        // Promote with an explicit mediaPlayback type (required/robust on
+        // targetSdk 34+); ServiceCompat handles the pre-Q signature. A throw
+        // here means the service never becomes foreground and the OS reclaims
+        // it on idle, so surface it loudly instead of swallowing.
+        try {
+            ServiceCompat.startForeground(
+                this,
+                NOTIFICATION_ID,
+                buildNotification(playbackState),
+                ServiceInfo.FOREGROUND_SERVICE_TYPE_MEDIA_PLAYBACK,
+            )
+            Log.d("MediaPlaybackService", "startForeground ok (state=$playbackState)")
+        } catch (e: Exception) {
+            Log.e("MediaPlaybackService", "startForeground failed", e)
+        }
    }

    private fun buildNotification(playbackState: Int): Notification {
@@ -326,56 +441,6 @@ class MediaPlaybackService : MediaBrowserServiceCompat() {
                    stopSelf(startId)
                }
            }
-            "UPDATE_METADATA" -> {
-                currentTitle = intent.getStringExtra("title") ?: currentTitle
-                currentArtist = intent.getStringExtra("artist") ?: currentArtist
-                // Unmarshal the artwork Bitmap off the main thread; copying its
-                // pixel buffer out of the Parcel can stall the UI thread and trip
-                // an ANR for large covers.
-                serviceScope.launch {
-                    val newArtwork = withContext(Dispatchers.Default) {
-                        if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.TIRAMISU) {
-                            intent.getParcelableExtra("artwork", Bitmap::class.java)
-                        } else {
-                            @Suppress("DEPRECATION")
-                            intent.getParcelableExtra("artwork")
-                        }
-                    }
-                    if (newArtwork != null) {
-                        currentArtwork = newArtwork
-                    }
-                    if (!isActive) return@launch
-                    val metadataBuilder = MediaMetadataCompat.Builder()
-                        .putString(MediaMetadataCompat.METADATA_KEY_TITLE, currentTitle)
-                        .putString(MediaMetadataCompat.METADATA_KEY_ARTIST, currentArtist)
-                        .putBitmap(MediaMetadataCompat.METADATA_KEY_ALBUM_ART, currentArtwork)
-                    mediaSession?.setMetadata(metadataBuilder.build())
-                    notifyChildrenChanged(MEDIA_ROOT_ID)
-                    if (sessionActive) {
-                        showNotification(if (player.isPlaying) PlaybackStateCompat.STATE_PLAYING else PlaybackStateCompat.STATE_PAUSED)
-                    }
-                }
-            }
-            "UPDATE_PLAYBACK_STATE" -> {
-                if (sessionActive) {
-                    val isPlaying = intent.getBooleanExtra("playing", false)
-                    val position = intent.getLongExtra("position", 0L) // in milliseconds
-                    val duration = intent.getLongExtra("duration", 0L) // in milliseconds
-
-                    if (isPlaying && !player.isPlaying) {
-                        player.play()
-                    } else if (!isPlaying && player.isPlaying) {
-                        player.pause()
-                    }
-                    player.seekTo(position)
-
-                    val state = if (isPlaying) PlaybackStateCompat.STATE_PLAYING else PlaybackStateCompat.STATE_PAUSED
-                    mediaSession?.setPlaybackState(
-                        stateBuilder.setState(state, position, 1f).build()
-                    )
-                    showNotification(state)
-                }
-            }
        }

        return super.onStartCommand(intent, flags, startId)
@@ -383,7 +448,6 @@ class MediaPlaybackService : MediaBrowserServiceCompat() {

    override fun onDestroy() {
        instance = null
-        serviceScope.cancel()
        super.onDestroy()
        player.release()
        mediaSession?.release()
@@ -95,7 +95,6 @@ class UpdateMediaSessionStateArgs {
@InvokeArg
 class SetMediaSessionActiveArgs {
  var active: Boolean? = null
-  var keepAppInForeground: Boolean? = null
  var notificationTitle: String? = null
  var notificationText: String? = null
  var foregroundServiceTitle: String? = null
@@ -497,19 +496,13 @@ class NativeTTSPlugin(private val activity: Activity) : Plugin(activity) {
        val args = invoke.parseArgs(UpdateMediaSessionMetadataArgs::class.java)
        val title = args.title ?: ""
        val artist = args.artist ?: ""
-        val album = args.album ?: ""

        coroutineScope.launch {
            try {
                val artworkBitmap = args.artwork?.let { loadArtworkFromUrl(it) }
-                val intent = Intent(activity, MediaPlaybackService::class.java).apply {
-                    action = "UPDATE_METADATA"
-                    putExtra("title", title)
-                    putExtra("artist", artist)
-                    putExtra("album", album)
-                    putExtra("artwork", artworkBitmap)
-                }
-                activity.startService(intent)
+                // In-process update on the running service; never startService()
+                // — that throws "app is in background" once backgrounded.
+                MediaPlaybackService.pushMetadata(title, artist, artworkBitmap)
                invoke.resolve()
            } catch (e: Exception) {
                invoke.reject("Failed to update metadata: ${e.message}")
@@ -519,19 +512,19 @@ class NativeTTSPlugin(private val activity: Activity) : Plugin(activity) {

    @Command
    fun update_media_session_state(invoke: Invoke) {
-        var args = invoke.parseArgs(UpdateMediaSessionStateArgs::class.java)
+        val args = invoke.parseArgs(UpdateMediaSessionStateArgs::class.java)
        val isPlaying = args.playing ?: false
-        val position = args.position ?: 0
-        val duration = args.duration ?: 0

        try {
-            val intent = Intent(activity, MediaPlaybackService::class.java).apply {
-                action = "UPDATE_PLAYBACK_STATE"
-                putExtra("playing", isPlaying)
-                putExtra("position", position)
-                putExtra("duration", duration)
-            }
-            activity.startService(intent)
+            // In-process update on the running service; never startService()
+            // — that throws "app is in background" once backgrounded. position
+            // and duration are null on a bare play/pause flip; the service
+            // keeps the last known values so the scrubber does not reset.
+            MediaPlaybackService.pushPlaybackState(
+                isPlaying,
+                args.position?.toLong(),
+                args.duration?.toLong(),
+            )
            invoke.resolve()
        } catch (e: Exception) {
            invoke.reject("Failed to update playback state: ${e.message}")
@@ -557,6 +550,7 @@ class NativeTTSPlugin(private val activity: Activity) : Plugin(activity) {
                val intent = Intent(activity, MediaPlaybackService::class.java).apply {
                    action = MediaPlaybackService.ACTION_ACTIVATE_SESSION
                }
+                Log.d(TAG, "set_media_session_active: startForegroundService")
                ContextCompat.startForegroundService(activity, intent)
            } else {
                // Not stopService: Android Auto may keep the service bound for