readest/apps/readest-app/.claude/memory/gdrive-fullwalk-every-sync-no-source-cursor.md

name, description, metadata

name	description	metadata
gdrive-fullwalk-every-sync-no-source-cursor	Google Drive file sync re-probes all 646 books every run (focus/Sync Now) because uploadedHashes never records no-source books; plus supabase focus events re-fire pullLibrary	node_type type originSessionId memory project 894e0d6d-ce01-402b-8f2d-0f0670986a88

Diagnosed 2026-07-07 (web dev, valid Drive session). "Uploading N / 646" on every tab refocus and every Sync Now = full per-book Drive probe sweep (files?q=name='<title>.epub' and '<hashdir>' in parents, ~1 req/book), no actual byte re-upload.

Two compounding causes:

1. File cursor never records books absent from this device. #4856's uploadedHashes in library.json is only added on uploaded or remote-matches (needs local bytes for size compare) in FileSyncEngine.syncLibrary push loop (engine.ts ~line 806-815). On web, non-downloaded books → loadBookFile null → no-source → NOT recorded even though the HEAD probe already proved the remote file exists. So needsFilePush stays true for all 646 forever → O(library) every run. Toggle test: Upload Book Files off → 15 reqs (config cursor isLocalNewer works); on → 646. Fix v1 (record remote-present no-source books, commit 900af1df1 on dev) proved INSUFFICIENT: Drive API inspection showed 654/690 hash dirs hold only cover.png+config.json, NO book file (only 36 files ever uploaded) — so there was nothing to record and the probe storm persisted. Fix v2 (the real fix, on dev 2026-07-07, initially uncommitted): reorder pushBookFile to resolve the LOCAL source before any remote probe (probeRemoteHead closure, lazy); no-source now costs zero requests and the remoteExists plumbing from v1 was removed again. Test: 'spends no remote request on a no-source book' in engine-sync-paths.test.ts. The 654 books stay in booksToPush (progress counter still shows them) but the sweep is network-free. Their files land on Drive only when a device that HAS the bytes (desktop) syncs with Upload Book Files on; that device records the hashes and everyone skips thereafter. Worktree was discarded per user; work continues directly on the bare repo dev branch (dev server localhost:3000 runs from there).

Round 5 (dev, uncommitted): Tauri plugin:fs|exists storm (5726 IPC/sync) killed by making the LIBRARY ROW the ground truth for local file presence in needsFilePush: gate hasLocalFile(b) = !!(b.downloadedAt || b.filePath) (import/download/delete all stamp downloadedAt; mergeBookMetadata keeps it device-local, verified) + session-scoped noSourceVerdicts WeakMap (provider-keyed, updatedAt-keyed) that suppresses re-probes of DRIFTED rows (row claims file, fs disagrees). Per user decision: NO automatic row-correction — Full Sync is the single split-brain healer (bypasses gate + memo + uploadedHashes and audits the real fs). Incremental sync with clean state = zero local and zero remote per-book probes. Harness in engine-sync-paths.test.ts: row-gate test, drifted-row memo tests, fullSync bypass; #4856 fixtures stamped downloadedAt.

Round 4 (dev, uncommitted): per-book cloud buttons (Book Details + bookshelf + open-non-local-book) route to the selected provider instead of the gated Readest Cloud queue ("Uploads to Readest Cloud are paused..." toast). FileSyncEngine.downloadBookFile (hash-dir listing resolves filename; stream on Tauri, buffered on web; cover+config best-effort) + runActiveFileBookUpload/Download in runLibrarySync.ts (stamps downloadedAt; caller persists via updateBook + toasts, existing transferMessages i18n keys). Reader hint parity same day: remoteProgressApplied in useFileSync dispatches 'Reading Progress Synced' hint on applied remote position. NOT done: provider path has no transfer-queue/progress UI; uploadedAt not stamped (means Readest-Cloud backup; provider uploaded-state could later key off index uploadedHashes).

Round 2 optimizations (dev, uncommitted as of 2026-07-07 03:10): provider memoized per connection key in createFileSyncProvider (warm Drive idCache across reader hook / library auto-sync / Sync Now; resetFileSyncProviderCache() called on Drive connect/disconnect); writeBinary PATCHes cached id without files.list lookup (404 evict+fallback); dev-only request diagnostics [gdrive] op ... / [gdrive] #n ... in GoogleDriveProvider. Remaining per-run budget after round 2 (no-change run ~11 req, ~550 kB): index GET 269 kB + index PATCH 269 kB every run; books/ listing 40 kB; ~8 file-less orphan hash dirs (in neither index nor library) re-listed by discovery every run. Round 3 (dev, uncommitted, all TDD in engine-sync-paths.test.ts): (1) etag short-circuit — remoteIndexCache WeakMap keyed on the memoized provider in engine.ts; head(library.json) etag (Drive md5/WebDAV ETag) vs cached → reuse structuredClone'd index, skip GET + ENTIRE discovery scan (peer changes always rewrite library.json; legacy no-index uploads still found on session-first run + fullSync); cache dropped after own push. (2) no-op push skip — indexDirty check (syncedHashes/failures/uploadedHashes-set/emptyDirs-set/any local row absent-or-newer-or-tombstone-mismatched vs remote index); skipping also keeps peers' etags stable (a restamped copy would defeat fleet-wide change detection). (3) emptyDirs optional index field (wire.ts) — file-less candidate dirs recorded once, skipped by discovery unless uploadedHashes says the file arrived or fullSync; pruned only against a listing that ran. Idle run = 1 stat request; local-change run = stat + config pull/push + index PATCH (no GET, no discovery). engine-deletion-sync 'preserves remote tombstone' test updated to force a dirty run.

2. Every tab focus re-runs the library file sync. supabase-js emits SIGNED_IN/TOKEN_REFRESHED on visibilitychange; AuthContext.syncSession does setUser(newObject) each time → pullLibrary (deps include user) recreated → useBooksSync effect [user, useSyncInited, libraryLoaded, pullLibrary] refires → runActiveFileLibrarySync (third-party provider path). Fix would be: key on user?.id / latch the initial pull. User decided 2026-07-07 to LEAVE THIS AS IS ("sync on focus is fine now that runs are O(changed)") — only cause 1 was fixed.

Related: #4981 fixed the adjacent expired-token variant (aborting instead of marching with remoteIndex=null). Web Drive token is sessionStorage-scoped (tab-local, no refresh). See cloud-sync-provider-selection-plan, webdav-filesync-refactor-plan.