ccb937015d
* fix(sync): record remote-present files for no-source books in the upload cursor With "Upload Book Files" on, a device that holds no local copy of a book (e.g. the web app with a cloud-only library) HEAD-probed the remote for every book on every sync: pushBookFile returned 'no-source' and the hash was never recorded in library.json's uploadedHashes, so needsFilePush stayed true for the whole library and each run (tab focus, Sync Now, library change) issued one Drive/WebDAV request per book, 646 requests per sweep in the reported case. The HEAD probe already answers whether the file is on the remote. Carry that in PushBookFileResult.remoteExists and record the hash when a no-source book's file is already mirrored, so the next incremental sync skips it and stays O(changed). Books absent both locally and remotely stay unrecorded so a device that has the bytes can upload them later. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(sync): reach the no-source verdict without probing the remote A book file sync with "Upload Book Files" on probed the remote for every book before checking whether this device even holds the bytes. On a device with a cloud-only library (the web app), none of the books have a local source, so every sync run (tab focus, Sync Now, library change) issued one name-lookup request per book against Google Drive, a full per-book request storm that never converged: with no local file there is nothing to upload and nothing to record, so the next run repeated it. Resolve the local source first and return 'no-source' from local state alone; the remote head probe now runs only when there is a local file to compare or upload. The probe keeps its non-NETWORK rethrow semantics so the auth-failure latch (#4981) still stops a run on an expired session. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(sync): incremental file sync and per-book transfers for the active provider Cut the redundant remote work a file-sync run does and route explicit per-book uploads/downloads to the active third-party provider (WebDAV / Google Drive) instead of the gated Readest Cloud transfer queue. Engine (services/sync/file/engine.ts, wire.ts): - etag change-probe: one HEAD on library.json, cached per provider for the session. When the etag matches the last successful pull, reuse the cached index and skip both the index download and the discovery scan. An AUTH failure on the probe aborts the run like the full pull does. - emptyDirs memo carried in the index: dirs found to hold no book file are recorded so clients stop re-listing them every run. Re-checked when the file arrives (uploadedHashes), on Full Sync, or when a legacy client drops the record; pruned only against a listing that actually ran. - skip the index re-push when the rebuilt index is semantically identical to the pulled one, so a restamped byte-copy no longer churns the remote and invalidates peers' etag change detection. - downloadBookFile() for the explicit per-book Download action. Provider reuse (services/sync/file/providerRegistry.ts): - memoise one provider per connection key, shared by every surface (reader per-book sync, library auto-sync, Sync now / pull to refresh). Reuses the Drive path->id cache instead of re-resolving /Readest, books/ and library.json by name query on every engine build. Drive connect/disconnect resets the cache since its token source changes identity with no key input changing. Google Drive (services/sync/providers/gdrive/GoogleDriveProvider.ts): - write fast-path: PATCH a path whose id is cached in place with no lookup, falling back to a full resolve on a 404 (stale id). Removes a files.list per PUT in the steady state. - dev-only request diagnostics: one line per provider op and per HTTP attempt so a run's request budget can be attributed from the console. Per-book transfers (services/sync/file/runLibrarySync.ts): - runActiveFileBookUpload / runActiveFileBookDownload build the active provider's engine and push/pull a single book, stamping downloadedAt like the native path. Wired into the reader/library book actions with toasts. UI, status, and i18n: - Readest Cloud sub-page: drop the quota stats and wrap the "Account and Storage" row in the BoxedList primitive so it aligns to the design system. - Shorten provider status/toast copy ("Active", "Google Drive session expired", "Library sync via {{provider}}", "KOReader") and translate the new keys across all locales. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
249 lines
9.5 KiB
TypeScript
249 lines
9.5 KiB
TypeScript
import clsx from 'clsx';
|
|
import React from 'react';
|
|
import { MdCloudSync } from 'react-icons/md';
|
|
import { v4 as uuidv4 } from 'uuid';
|
|
import { useEnv } from '@/context/EnvContext';
|
|
import { useTranslation, type TranslationFunc } from '@/hooks/useTranslation';
|
|
import { useSettingsStore } from '@/store/settingsStore';
|
|
import { useLibraryStore } from '@/store/libraryStore';
|
|
import { useFileSyncStore } from '@/store/fileSyncStore';
|
|
import { eventDispatcher } from '@/utils/event';
|
|
import { FileSyncEngine } from '@/services/sync/file/engine';
|
|
import { FileSyncError } from '@/services/sync/file/provider';
|
|
import { createAppLocalStore } from '@/services/sync/file/appLocalStore';
|
|
import {
|
|
createFileSyncProvider,
|
|
type FileSyncBackendKind,
|
|
} from '@/services/sync/file/providerRegistry';
|
|
import type { KOSyncStrategy } from '@/types/settings';
|
|
import { BoxedList, SettingsRow, SettingsSelect, SettingsSwitchRow } from '../primitives';
|
|
|
|
/** The settings fields the shared sync controls read/write (WebDAV + Drive share these). */
|
|
export interface FileSyncFormSettings {
|
|
enabled?: boolean;
|
|
syncBooks?: boolean;
|
|
fullSync?: boolean;
|
|
strategy?: KOSyncStrategy;
|
|
deviceId?: string;
|
|
lastSyncedAt?: number;
|
|
}
|
|
|
|
interface FileSyncFormProps {
|
|
/** Which backend these controls drive (also keys the progress store + mutex). */
|
|
kind: FileSyncBackendKind;
|
|
/** This backend's settings slice. */
|
|
stored: FileSyncFormSettings;
|
|
/** Persist a patch into this backend's settings slice (must merge store-latest). */
|
|
persist: (patch: Partial<FileSyncFormSettings>) => Promise<void>;
|
|
/**
|
|
* Disable the "Sync now" button — set when the connection needs attention
|
|
* (e.g. an expired web Google Drive session) so a manual sync that would just
|
|
* fail isn't offered. The parent panel shows the reconnect affordance.
|
|
*/
|
|
syncNowDisabled?: boolean;
|
|
}
|
|
|
|
/**
|
|
* Translate a sync-time error into a user-facing string. Backend-neutral: the
|
|
* provider maps every failure to a {@link FileSyncError} with a normalised `code`
|
|
* so we never show a raw English `e.message`.
|
|
*/
|
|
const formatSyncError = (_: TranslationFunc, e: unknown): string => {
|
|
if (e instanceof FileSyncError) {
|
|
switch (e.code) {
|
|
case 'AUTH_FAILED':
|
|
return _('Authentication failed. Reconnect in Settings.');
|
|
case 'NOT_FOUND':
|
|
return _('Remote resource not found');
|
|
case 'NETWORK':
|
|
return _('Network error');
|
|
}
|
|
if (typeof e.status === 'number') {
|
|
return _('Sync failed (status {{status}})', { status: e.status });
|
|
}
|
|
}
|
|
return _('Sync failed.');
|
|
};
|
|
|
|
/**
|
|
* The provider-agnostic sync controls shared by every file-sync backend's
|
|
* settings form: the sub-category toggles, the conflict strategy, and a manual
|
|
* "Sync now" button with progress + result toast. The backend-specific connect
|
|
* panel (WebDAV URL/credentials, the Drive Connect button) lives in the parent
|
|
* form; everything below the connect line is identical across backends, so it
|
|
* lives here once and is parameterised by {@link FileSyncFormProps.kind}.
|
|
*/
|
|
const FileSyncForm: React.FC<FileSyncFormProps> = ({
|
|
kind,
|
|
stored,
|
|
persist,
|
|
syncNowDisabled = false,
|
|
}) => {
|
|
const _ = useTranslation();
|
|
const { settings } = useSettingsStore();
|
|
const { envConfig } = useEnv();
|
|
|
|
const isSyncing = useFileSyncStore((s) => s.byKind[kind]?.isSyncing ?? false);
|
|
const syncProgressLabel = useFileSyncStore((s) => s.byKind[kind]?.progressLabel ?? null);
|
|
const syncProgressDetail = useFileSyncStore((s) => s.byKind[kind]?.progressDetail ?? null);
|
|
const beginSync = useFileSyncStore((s) => s.beginSync);
|
|
const updateProgress = useFileSyncStore((s) => s.updateProgress);
|
|
const endSync = useFileSyncStore((s) => s.endSync);
|
|
|
|
const handleToggleSyncBooks = () => persist({ syncBooks: !(stored.syncBooks ?? false) });
|
|
const handleToggleFullSync = () => persist({ fullSync: !(stored.fullSync ?? false) });
|
|
const handleStrategyChange = async (e: React.ChangeEvent<HTMLSelectElement>) => {
|
|
await persist({ strategy: e.target.value as KOSyncStrategy });
|
|
};
|
|
|
|
/**
|
|
* Manual "Sync now" — reconcile the local library with the remote over a
|
|
* bounded-concurrency pool. Incremental by default (only books whose local
|
|
* copy differs from the shared index); "Full Sync" re-checks every book. The
|
|
* provider is built by kind through the registry so this stays backend-neutral.
|
|
*/
|
|
const handleSyncNow = async () => {
|
|
if (syncNowDisabled) return;
|
|
if (useFileSyncStore.getState().byKind[kind]?.isSyncing) return;
|
|
if (!stored.enabled) return;
|
|
|
|
const { libraryLoaded, library } = useLibraryStore.getState();
|
|
const appService = await envConfig.getAppService();
|
|
|
|
let currentLibrary = library ?? [];
|
|
if (!libraryLoaded && appService) {
|
|
currentLibrary = await appService.loadLibraryBooks();
|
|
// Hydrate the store before syncing so the engine's addBookToLibrary /
|
|
// updateBookMetadata merge against the real library, not an empty one.
|
|
useLibraryStore.getState().setLibrary(currentLibrary);
|
|
}
|
|
|
|
// Count only live books for the progress label, but sync the FULL library
|
|
// (including soft-deleted books): the engine tombstones deleted books in
|
|
// library.json so deletions propagate, and keeping them in the input set
|
|
// stops the discovery pass from re-downloading a book the user just deleted.
|
|
const liveBookCount = currentLibrary.filter((b) => !b.deletedAt).length;
|
|
|
|
// Lazily ensure a deviceId so the first cross-device sync attributes its
|
|
// rows correctly (the reader hook also touches this on first push).
|
|
let deviceId = stored.deviceId;
|
|
if (!deviceId) {
|
|
deviceId = uuidv4();
|
|
await persist({ deviceId });
|
|
}
|
|
|
|
// Acquire the global library-sync mutex; bail if another backend's Sync now
|
|
// is already mutating the local library.
|
|
if (!beginSync(kind, _('Syncing {{n}} / {{total}}', { n: 0, total: liveBookCount }))) {
|
|
return;
|
|
}
|
|
|
|
try {
|
|
const provider = await createFileSyncProvider(kind, settings);
|
|
if (!provider) {
|
|
throw new FileSyncError('Sync backend is not available on this device', 'UNKNOWN');
|
|
}
|
|
const store = createAppLocalStore({ appService, settings, envConfig });
|
|
const engine = new FileSyncEngine(provider, store);
|
|
const result = await engine.syncLibrary(currentLibrary, {
|
|
strategy: stored.strategy === 'prompt' ? 'silent' : stored.strategy,
|
|
syncBooks: stored.syncBooks ?? false,
|
|
fullSync: stored.fullSync ?? false,
|
|
deviceId: deviceId as string,
|
|
onProgress: ({ book, index, total, action }) => {
|
|
const actionStr = action === 'downloading' ? _('Downloading') : _('Uploading');
|
|
updateProgress(
|
|
kind,
|
|
_('{{action}} {{n}} / {{total}}', { action: actionStr, n: index + 1, total }),
|
|
book.title || book.hash.slice(0, 8),
|
|
);
|
|
},
|
|
});
|
|
|
|
await persist({ lastSyncedAt: Date.now() });
|
|
if (result.failures > 0) {
|
|
eventDispatcher.dispatch('toast', {
|
|
type: 'warning',
|
|
message: _('Sync finished with {{failed}} failure(s). {{ok}} ok.', {
|
|
failed: result.failures,
|
|
ok: Math.max(0, result.totalBooks - result.failures),
|
|
}),
|
|
});
|
|
} else {
|
|
eventDispatcher.dispatch('toast', {
|
|
type: 'info',
|
|
message: _('{{count}} book(s) synced', { count: result.booksSynced }),
|
|
});
|
|
}
|
|
} catch (e) {
|
|
eventDispatcher.dispatch('toast', { type: 'error', message: formatSyncError(_, e) });
|
|
} finally {
|
|
endSync(kind);
|
|
}
|
|
};
|
|
|
|
return (
|
|
<BoxedList>
|
|
<SettingsSwitchRow
|
|
label={_('Upload Book Files')}
|
|
description={_('Uploads book files to your other devices')}
|
|
checked={stored.syncBooks ?? false}
|
|
onChange={handleToggleSyncBooks}
|
|
/>
|
|
<SettingsSwitchRow
|
|
label={_('Full Sync')}
|
|
description={_('Re-check every book instead of only changed ones')}
|
|
checked={stored.fullSync ?? false}
|
|
onChange={handleToggleFullSync}
|
|
/>
|
|
<SettingsRow label={_('Sync Strategy')}>
|
|
<SettingsSelect
|
|
value={stored.strategy ?? 'silent'}
|
|
onChange={handleStrategyChange}
|
|
ariaLabel={_('Sync Strategy')}
|
|
options={[
|
|
{ value: 'silent', label: _('Send and receive') },
|
|
{ value: 'send', label: _('Send only') },
|
|
{ value: 'receive', label: _('Receive only') },
|
|
]}
|
|
/>
|
|
</SettingsRow>
|
|
<SettingsRow
|
|
label={
|
|
syncProgressLabel
|
|
? syncProgressLabel
|
|
: stored.lastSyncedAt
|
|
? _('Last synced {{when}}', { when: new Date(stored.lastSyncedAt).toLocaleString() })
|
|
: _('Never synced')
|
|
}
|
|
description={
|
|
syncProgressDetail ? (
|
|
<span className='line-clamp-1'>{syncProgressDetail}</span>
|
|
) : undefined
|
|
}
|
|
>
|
|
<button
|
|
type='button'
|
|
onClick={handleSyncNow}
|
|
disabled={isSyncing || syncNowDisabled}
|
|
className={clsx(
|
|
'btn btn-ghost btn-sm h-8 min-h-8 gap-1 px-2',
|
|
(isSyncing || syncNowDisabled) && 'opacity-60',
|
|
)}
|
|
title={_('Sync now')}
|
|
aria-label={_('Sync now')}
|
|
>
|
|
{isSyncing ? (
|
|
<span className='loading loading-spinner loading-xs' />
|
|
) : (
|
|
<MdCloudSync className='h-4 w-4' />
|
|
)}
|
|
{_('Sync now')}
|
|
</button>
|
|
</SettingsRow>
|
|
</BoxedList>
|
|
);
|
|
};
|
|
|
|
export default FileSyncForm;
|