forked from akai/readest
f805477091babdfd6783671dfbe7aea7a47e0980
6 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Huang Xin
|
f805477091 |
perf(koplugin): defer and cache Library group covers (#4954) (#4974)
Opening the Library with a large grouped library was slow because each folder's 2x2 cover mosaic was recomposed from scratch on every paint (up to 4 MuPDF cover decodes plus scales per cell). On a 685-book library this dominated the synchronous open path (254ms of 300ms) and ran again on the post-sync refresh. Navigation felt fast only because drilling into a group shows single covers, not mosaics. Mirror cloud_covers' async pattern in group_covers: - Cache the composed master bb per group, keyed by a signature that flips when the child set or any child's cover availability changes; serve cheap copies on a hit. Cache a nil result too, so a group whose covers are not ready yet keeps its placeholder without recomposing on every refresh; a later cover download flips the signature and recomposes once. - Compose off the first-paint path: a miss enqueues a background job (one mosaic per UI tick) and returns nil so the cell paints its FakeCover placeholder immediately; finished mosaics coalesce into one refresh. - Free cached masters when the Library closes. Measured synchronous open path drops from 300ms to 151ms on a 685-book library; mosaic compositing moves off the blocking paint and fills in progressively. Adds open-path timing logs to librarywidget and localscanner for on-device diagnosis of future large-library reports. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
e0b537bc16 |
feat(koplugin): bulk download all cloud books from Library view (#4751) (#4765)
Downloading a Readest cloud library into KOReader previously required tapping each book one at a time. Add a "Download all books" action to the Library view menu that pulls every cloud-only book to the device in one pass. - LibraryStore:listCloudOnlyBooks() returns the downloadable cloud-present, not-local books for the current user (whole library, independent of the active search/group view). - librarywidget.downloadAll() streams them sequentially via the existing syncbooks.downloadBook path inside a Trapper coroutine: cancellable progress (Trapper:info Abort/Continue), per-book failures skipped and counted, summary toast at the end. Bridges downloadBook's sync-or-async callback with a coroutine suspended check so it serializes correctly either way. - Wire the action into the view-menu Actions section. - Add the new UI strings and translate them across all 33 locales. Closes #4751 Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
72233e1c6a |
feat: sync reading status across devices and with KOReader (#4634) (#4656)
* docs(sync): design spec for syncing reading status (#4634) Field-level LWW for reading_status (dedicated reading_status_updated_at), a new 'abandoned' status in the Readest UI, and a koplugin bridge to KOReader's native summary.status (whole-library apply + capture). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(sync): implementation plan for syncing reading status (#4634) Bite-sized TDD tasks across 3 parts: A) cloud field-level LWW (reading_status_updated_at on server upsert + client pull-merge), B) 'abandoned'/On-hold status in the Readest UI, C) koplugin bridge to KOReader summary.status (mapping + reconcile + whole-library apply/capture). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(sync): add reading_status_updated_at for field-level status LWW (#4634) * feat(sync): stamp readingStatusUpdatedAt on status change in updateBookProgress Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(sync): stamp status timestamp on explicit library status edits * feat(sync): resolve reading status by its own timestamp in client pull-merge * feat(sync): resolve reading status by its own timestamp in server upsert (#4634) * fix(sync): tighten reading-status merge typing + strengthen test (A5 review) Replace as-unknown-as double-casts at read sites with typed locals (clientBook/serverBook); retain a single as-unknown-as only at the server-wins construction site where the static type is too narrow. Strengthen test 3 to assert both fields with toEqual. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(library): render the 'On hold' (abandoned) status badge * feat(library): add 'Mark as On hold' actions + i18n for abandoned status Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * i18n: translate 'On hold' and 'Mark as On hold' for the abandoned status (#4634) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(koplugin): add reading-status mapping + reconcile between Readest and KOReader * feat(koplugin): persist + sync reading_status_updated_at in LibraryStore Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(koplugin): bridge reading status to KOReader summary.status on library sync (#4634) * test(sync): cover koplugin v1->v2 migration + tighten status-sync tests (final review) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(sync): redesign KOReader first-sync (decisive-only + bootstrap) (#4634) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(koplugin): safe first-sync of reading status (decisive-only + bootstrap) (#4634) KOReader auto-sets summary.status='reading' on first open, and legacy Readest statuses have reading_status_updated_at=0, so pure timestamp LWW let opening a finished book downgrade it. Restrict sync to deliberate statuses (finished/ complete, abandoned/on-hold, unread->clear); never capture KO 'reading'/'New'. On the unsynced baseline (Readest ts=0) conflicts resolve Readest-authoritative, then stamp now_ms to exit bootstrap into steady-state LWW. reconcile now returns write_ko/write_store flags; statussync captures now_ms once and equalizes both sides (convergent, idempotent, resumable). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(koplugin): cover remaining first-sync graph cells + document sort effect (review) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Huang Xin
|
ab2def32dd |
fix(koplugin): stop sync from wiping cloud book fields + Library polish (#4166)
* fix(koplugin): pull before push so sync doesn't wipe cloud book fields, closes #4138 The Library sync pushed a touched book row before pulling, so a row still missing the cloud's uploaded_at / metadata / group_id (e.g. one created by lightScan, not yet merged from a cloud pull) was sent with those fields nil. The server's transformBookToDB explicit-nulls uploaded_at and metadata for any field absent from the wire payload, wiping the cloud copy — after which every device that pulled lost the book's upload state. syncBooks("both") now pulls first, then pushes, and takes a before_push callback. syncBooksLibrary passes touchOpenBook through it so the open book's updated_at bump lands after the pull has refreshed the local row, letting the push carry the preserved cloud fields. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(koplugin): hide Library books with neither an uploaded nor a local file The Library showed any row with cloud_present = 1, but a bare cloud *record* whose file was never uploaded (uploaded_at NULL) has no cover and can't be opened — showing it is meaningless. Tighten the visibility predicate to (uploaded_at IS NOT NULL OR local_present = 1) across listBooks, getGroups, listBookshelfGroups and listBooksInGroup. This mirrors Readest, which only adds a synced book to the library when uploadedAt is set and keeps locally-imported books that carry a downloadedAt. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(koplugin): close the Library widget when opening a book Opening a book from the Library called ReaderUI:showReader without closing the Library Menu, so it stayed in the UIManager widget stack with M._menu still set. A later background M.refresh() — a cloud-sync or cover-download completion — then repainted that ghost Library over the reader, making it flash on screen for a few seconds. Add M.close(); route the title-bar X, M.reopen() and both handleTap book-open paths through it. A wrapped onCloseWidget clears M._menu on every close path, so M.refresh() no-ops once the Menu is gone. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Jon Volkmar
|
f6b6281160 | fix(kosync): add namespace to koreader plugin modules to avoid collision (#4153) | ||
|
Huang Xin
|
ed8956e9ed |
feat(koplugin): Readest Library view in KOReader (#4056)
* feat(koplugin/library): data layer + busted harness + design doc - LibraryStore: per-user SQLite index merging cloud + local books by partial-md5 hash. listBooks/listBookshelfBooks/listBookshelfGroups, upsertBook with cloud_present/local_present OR-merge + _force/clear sentinels, parseSyncRow, getChangedBooks for tombstone push. - EXTS table mirroring web's document.ts. - busted harness with KOReader stubs (G_reader_settings, DataStorage, lua-ljsqlite3 against :memory:); spec_helper, exts_spec, smoke_spec, librarystore_spec covering schema, sort, group nesting, dedupe. - Library design doc + spec README. - pnpm test:lua wired through root + app package.json; lint-koplugin recurses into library/ + spec/. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(koplugin/library): cloud sync — push, pull, upload, delete, downloads - Spore methods: pullBooks (incremental /sync), getDownloadUrl, getUploadUrl, listFiles, deleteFile. - syncbooks.lua: pushBook + pushChangedBooks (advances watermark to max(updated_at, deleted_at)), syncBooks(opts, mode) for push/pull/both, downloadBook + downloadCover (sync socket.http with file sink; cover download via fork+poll with single-slot queue + visible-page filter + coalesced refresh), uploadBook (presigned-PUT flow + best-effort cover), deleteCloudFiles (list-then-delete-each, mirrors cloudService.deleteBook). - SyncAuth.withFreshToken wrapper resolves the ensureClient race; 401/403 unified across syncconfig + syncannotations. - Cloud + local book covers shared by partial-md5 hash; cover.png cached at <settings>/readest_covers/<hash>.png with sentinel for known 404s. - syncbooks_spec covers row-to-wire conversion + file_key shape. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(koplugin/library): local discovery — sidecar scanner + cover provider - localscanner.lightScan iterates ReadHistory entries, reads partial_md5_checksum from .sdr sidecars, and upserts local rows. Slow filesystem walks deferred to fullSidecarWalk (24h-gated). - coverprovider wraps BookInfoManager:getBookInfo for local books with graceful FakeCover fallback when coverbrowser is absent. - localscanner_spec + coverprovider_spec. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(koplugin/library): UI — widget, item, view menu, FileManager hooks - librarywidget: full-screen Menu mixed in with CoverMenu + Mosaic/List per zen_ui's group_view pattern. Title-bar tap → view menu, search via left icon, drill-in/back for grouping. Async cloud sync deferred via scheduleIn so the menu paints before HTTP fires. - libraryitem: BIM patch for cloud-only (readest-cloud://) and group (readest-group://) URIs; group-cover composer (2x2 mosaic for grid, same in list) with cache key derived from the actual first-N hashes for content-based invalidation; ListMenuItem update + paintTo patches for wider list-mode cover strip and cloud-up/down icon overlay. - libraryviewmenu: ButtonDialog with View/Group by/Sort by/Actions. Default Group by = Groups (parity with web), values authors/groups. - librarypaint: partial-page e-ink repaint shim adapted from zen_ui. - main.lua: Library menu entry, dispatcher actions (Open Library / Push / Pull as general; progress + annotations as reader-only), "Add to Readest" button in FileManager's long-press file dialog (dedupe by partial_md5; bumps updated_at when present, inserts a fresh local-only row otherwise; un-tombstones via _clear_fields). Push books on Library open when auto_sync is on, pull-only otherwise. - Long-press action sheet with Readest BookDetailView parity: Remove from Cloud & Device / Cloud Only / Device Only, Upload to Cloud, Download Book / Cover / All. - Cloud-down + cloud-up SVG icons (LiaCloudDownloadAltSolid / LiaCloudUploadAltSolid) painted in the right-side wpageinfo slot. - i18n catalog updated for new strings. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(koplugin/library): split libraryitem into focused modules libraryitem.lua had grown to 1018 lines mixing five unrelated concerns. Split along the natural seams: cloud_covers — readest-cloud:// URI scheme, on-disk <hash>.png cache, single-slot async download queue, visible-page filter group_covers — readest-group:// URI scheme, 2x2 mosaic composer with content-derived cache key (first-N hashes), cell layout table cloud_icons — bundled cloud-up/cloud-down SVG loader, IconWidget cache, paint-overlay positioning list_strip — list-mode group row builder (4-cover wider strip replacing ListMenu's square cover slot) bim_patch — BookInfoManager:getBookInfo router (cloud / group / local) + ListMenuItem update + paintTo patches; owns the _library_local_paths set and orig BIM reference libraryitem.lua is now 141 lines: just the entry-table constructors (entry_from_row, entry_from_group, entry_back) plus thin install / set_visible_hashes delegates. Each new module is 88-216 lines. No behavior change — same 113 specs pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(koplugin): co-locate dev tooling, ship-zip exclusions, CI job split - apps/readest.koplugin/scripts/build-koplugin.mjs: local sideloading build with the same exclusions the release workflow uses. - Move lint-koplugin + test-koplugin from apps/readest-app/scripts/ to apps/readest.koplugin/scripts/. All koplugin dev tooling now lives with the koplugin and is excluded from the published release zip. - Rename to .mjs so Node treats them as ESM without the reparse warning (the i18n CommonJS scripts stay .js). - Release workflow: zip -r exclusions for scripts/, docs/, spec/, .busted so dev artifacts don't ship to end users. - PR workflow: split build_web_app into build_web_app + test_web_app for parallelism. The test job installs luarocks + busted + lsqlite3complete and runs pnpm test:lua. test-koplugin.mjs now hard-fails (instead of soft-skipping) when CI=true and a tool is missing — a broken CI toolchain previously exited 0 silently. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |