ed8956e9ed
* feat(koplugin/library): data layer + busted harness + design doc - LibraryStore: per-user SQLite index merging cloud + local books by partial-md5 hash. listBooks/listBookshelfBooks/listBookshelfGroups, upsertBook with cloud_present/local_present OR-merge + _force/clear sentinels, parseSyncRow, getChangedBooks for tombstone push. - EXTS table mirroring web's document.ts. - busted harness with KOReader stubs (G_reader_settings, DataStorage, lua-ljsqlite3 against :memory:); spec_helper, exts_spec, smoke_spec, librarystore_spec covering schema, sort, group nesting, dedupe. - Library design doc + spec README. - pnpm test:lua wired through root + app package.json; lint-koplugin recurses into library/ + spec/. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(koplugin/library): cloud sync — push, pull, upload, delete, downloads - Spore methods: pullBooks (incremental /sync), getDownloadUrl, getUploadUrl, listFiles, deleteFile. - syncbooks.lua: pushBook + pushChangedBooks (advances watermark to max(updated_at, deleted_at)), syncBooks(opts, mode) for push/pull/both, downloadBook + downloadCover (sync socket.http with file sink; cover download via fork+poll with single-slot queue + visible-page filter + coalesced refresh), uploadBook (presigned-PUT flow + best-effort cover), deleteCloudFiles (list-then-delete-each, mirrors cloudService.deleteBook). - SyncAuth.withFreshToken wrapper resolves the ensureClient race; 401/403 unified across syncconfig + syncannotations. - Cloud + local book covers shared by partial-md5 hash; cover.png cached at <settings>/readest_covers/<hash>.png with sentinel for known 404s. - syncbooks_spec covers row-to-wire conversion + file_key shape. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(koplugin/library): local discovery — sidecar scanner + cover provider - localscanner.lightScan iterates ReadHistory entries, reads partial_md5_checksum from .sdr sidecars, and upserts local rows. Slow filesystem walks deferred to fullSidecarWalk (24h-gated). - coverprovider wraps BookInfoManager:getBookInfo for local books with graceful FakeCover fallback when coverbrowser is absent. - localscanner_spec + coverprovider_spec. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(koplugin/library): UI — widget, item, view menu, FileManager hooks - librarywidget: full-screen Menu mixed in with CoverMenu + Mosaic/List per zen_ui's group_view pattern. Title-bar tap → view menu, search via left icon, drill-in/back for grouping. Async cloud sync deferred via scheduleIn so the menu paints before HTTP fires. - libraryitem: BIM patch for cloud-only (readest-cloud://) and group (readest-group://) URIs; group-cover composer (2x2 mosaic for grid, same in list) with cache key derived from the actual first-N hashes for content-based invalidation; ListMenuItem update + paintTo patches for wider list-mode cover strip and cloud-up/down icon overlay. - libraryviewmenu: ButtonDialog with View/Group by/Sort by/Actions. Default Group by = Groups (parity with web), values authors/groups. - librarypaint: partial-page e-ink repaint shim adapted from zen_ui. - main.lua: Library menu entry, dispatcher actions (Open Library / Push / Pull as general; progress + annotations as reader-only), "Add to Readest" button in FileManager's long-press file dialog (dedupe by partial_md5; bumps updated_at when present, inserts a fresh local-only row otherwise; un-tombstones via _clear_fields). Push books on Library open when auto_sync is on, pull-only otherwise. - Long-press action sheet with Readest BookDetailView parity: Remove from Cloud & Device / Cloud Only / Device Only, Upload to Cloud, Download Book / Cover / All. - Cloud-down + cloud-up SVG icons (LiaCloudDownloadAltSolid / LiaCloudUploadAltSolid) painted in the right-side wpageinfo slot. - i18n catalog updated for new strings. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(koplugin/library): split libraryitem into focused modules libraryitem.lua had grown to 1018 lines mixing five unrelated concerns. Split along the natural seams: cloud_covers — readest-cloud:// URI scheme, on-disk <hash>.png cache, single-slot async download queue, visible-page filter group_covers — readest-group:// URI scheme, 2x2 mosaic composer with content-derived cache key (first-N hashes), cell layout table cloud_icons — bundled cloud-up/cloud-down SVG loader, IconWidget cache, paint-overlay positioning list_strip — list-mode group row builder (4-cover wider strip replacing ListMenu's square cover slot) bim_patch — BookInfoManager:getBookInfo router (cloud / group / local) + ListMenuItem update + paintTo patches; owns the _library_local_paths set and orig BIM reference libraryitem.lua is now 141 lines: just the entry-table constructors (entry_from_row, entry_from_group, entry_back) plus thin install / set_visible_hashes delegates. Each new module is 88-216 lines. No behavior change — same 113 specs pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(koplugin): co-locate dev tooling, ship-zip exclusions, CI job split - apps/readest.koplugin/scripts/build-koplugin.mjs: local sideloading build with the same exclusions the release workflow uses. - Move lint-koplugin + test-koplugin from apps/readest-app/scripts/ to apps/readest.koplugin/scripts/. All koplugin dev tooling now lives with the koplugin and is excluded from the published release zip. - Rename to .mjs so Node treats them as ESM without the reparse warning (the i18n CommonJS scripts stay .js). - Release workflow: zip -r exclusions for scripts/, docs/, spec/, .busted so dev artifacts don't ship to end users. - PR workflow: split build_web_app into build_web_app + test_web_app for parallelism. The test job installs luarocks + busted + lsqlite3complete and runs pnpm test:lua. test-koplugin.mjs now hard-fails (instead of soft-skipping) when CI=true and a tool is missing — a broken CI toolchain previously exited 0 silently. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
214 lines
8.1 KiB
Lua
214 lines
8.1 KiB
Lua
-- cloud_covers.lua
|
|
-- Per-book cover lifecycle for cloud-only rows. Owns the on-disk
|
|
-- <hash>.png cache, the synthetic readest-cloud:// URI scheme, and
|
|
-- the single-slot async download queue that fetches missing covers
|
|
-- from Readest storage when their cells become visible.
|
|
--
|
|
-- The disk cache is shared with hybrid (cloud+local) rows so a
|
|
-- previously-downloaded cloud cover gets reused for the local
|
|
-- presentation of the same hash without a re-extraction pass.
|
|
|
|
local logger = require("logger")
|
|
|
|
local M = {}
|
|
|
|
M.URI_PREFIX = "readest-cloud://"
|
|
|
|
-- Synthetic-info metadata cache: keyed by either hash (cloud-only
|
|
-- entries) or the full URI string (group entries). Values:
|
|
-- { title, author }
|
|
-- The patched BIM reads these to fill in title/authors so MosaicMenu's
|
|
-- FakeCover renders meaningful text instead of "?".
|
|
local _meta = {}
|
|
|
|
-- Download lifecycle state
|
|
local _cover_pending = {} -- hash → true while a download is in flight
|
|
local _missing_covers = {} -- hash → true after a 404 (don't keep retrying)
|
|
local _visible_hashes = nil -- set of hashes on the current Menu page; nil = no filter
|
|
local _refresh_pending = false -- coalesces multiple cover-completion refreshes
|
|
local _download_queue = {} -- FIFO list of pending hashes
|
|
local _downloading = false -- gate: only one socket.http active at a time
|
|
|
|
-- Sync-auth opts (set via M.set_opts at install time). Holds
|
|
-- sync_auth, sync_path, settings — needed to drive syncbooks.downloadCover.
|
|
local _opts = nil
|
|
|
|
function M.set_opts(opts)
|
|
_opts = opts
|
|
end
|
|
|
|
function M.set_meta(key, meta)
|
|
_meta[key] = meta
|
|
end
|
|
|
|
function M.get_meta(key)
|
|
return _meta[key] or {}
|
|
end
|
|
|
|
function M.covers_dir()
|
|
local DataStorage = require("datastorage")
|
|
return DataStorage:getSettingsDir() .. "/readest_covers"
|
|
end
|
|
|
|
local function cover_path_for(hash)
|
|
return M.covers_dir() .. "/" .. hash .. ".png"
|
|
end
|
|
|
|
-- Strip the .ext suffix so callers get just the partial-md5. The .ext
|
|
-- is encoded into the URI so listmenu's filemanagerutil.splitFileNameType
|
|
-- returns a non-nil filetype (listmenu.lua:316); without it the
|
|
-- right-column composition crashes on string concat.
|
|
function M.hash_from_uri(filepath)
|
|
local rest = filepath:sub(#M.URI_PREFIX + 1)
|
|
return (rest:match("^([^.]+)") or rest)
|
|
end
|
|
|
|
-- Load <hash>.png from disk into a fresh blitbuffer. Returns nil if the
|
|
-- file doesn't exist or fails to decode. Caller owns the bb (ImageWidget
|
|
-- will free it). No in-memory cache — ImageWidget treats the bb as
|
|
-- disposable, so sharing one across paints leads to use-after-free.
|
|
function M.load_cover_bb(hash)
|
|
local lfs = require("libs/libkoreader-lfs")
|
|
local path = cover_path_for(hash)
|
|
if lfs.attributes(path, "mode") ~= "file" then return nil end
|
|
local ok, RenderImage = pcall(require, "ui/renderimage")
|
|
if not ok then return nil end
|
|
local ok2, bb = pcall(RenderImage.renderImageFile, RenderImage, path, false)
|
|
if not ok2 or not bb then return nil end
|
|
return bb
|
|
end
|
|
|
|
-- "<hash8> '<title>'" formatted log tag — searchable by either id.
|
|
local function tag_for(hash)
|
|
local meta = _meta[hash] or {}
|
|
return hash:sub(1, 8) .. " '" .. tostring(meta.title or "?") .. "'"
|
|
end
|
|
|
|
-- Pump the next entry off _download_queue. Re-entrant-safe via the
|
|
-- _downloading gate. Filters known-404 hashes and hashes that have
|
|
-- scrolled off-screen since they were enqueued.
|
|
local function process_queue()
|
|
if _downloading then return end
|
|
local hash
|
|
repeat
|
|
hash = table.remove(_download_queue, 1)
|
|
if not hash then return end
|
|
if _missing_covers[hash] then
|
|
_cover_pending[hash] = nil
|
|
hash = nil
|
|
elseif _visible_hashes and not _visible_hashes[hash] then
|
|
logger.dbg("ReadestLibrary cover dequeue skip: " .. tag_for(hash)
|
|
.. " no longer on visible page")
|
|
_cover_pending[hash] = nil
|
|
hash = nil
|
|
end
|
|
until hash
|
|
|
|
_downloading = true
|
|
logger.info("ReadestLibrary cover download: starting " .. tag_for(hash))
|
|
local syncbooks = require("library.syncbooks")
|
|
syncbooks.downloadCover(
|
|
{ hash = hash },
|
|
{
|
|
sync_auth = _opts and _opts.sync_auth,
|
|
sync_path = _opts and _opts.sync_path,
|
|
settings = _opts and _opts.settings,
|
|
covers_dir = M.covers_dir(),
|
|
},
|
|
function(success, path_or_err, status)
|
|
_cover_pending[hash] = nil
|
|
_downloading = false
|
|
if not success then
|
|
if status == 404 then
|
|
_missing_covers[hash] = true
|
|
logger.info("ReadestLibrary cover " .. tag_for(hash)
|
|
.. " — no cover on server (404), won't retry")
|
|
else
|
|
logger.warn("ReadestLibrary cover " .. tag_for(hash)
|
|
.. " download failed: " .. tostring(path_or_err)
|
|
.. " status=" .. tostring(status))
|
|
end
|
|
else
|
|
logger.info("ReadestLibrary cover " .. tag_for(hash)
|
|
.. " saved → " .. tostring(path_or_err))
|
|
-- Coalesce refresh: multiple covers landing in the same
|
|
-- tick still get one repaint, not N flickering redraws.
|
|
if not _refresh_pending then
|
|
_refresh_pending = true
|
|
local UIManager = require("ui/uimanager")
|
|
UIManager:nextTick(function()
|
|
_refresh_pending = false
|
|
local ok, LibraryWidget = pcall(require, "library.librarywidget")
|
|
if ok and LibraryWidget._menu then LibraryWidget.refresh() end
|
|
end)
|
|
end
|
|
end
|
|
-- Yield to the UI loop before pumping the next one.
|
|
local UIManager = require("ui/uimanager")
|
|
UIManager:nextTick(process_queue)
|
|
end)
|
|
end
|
|
|
|
-- Idempotent against in-flight requests and known-404 hashes. Only
|
|
-- fires for hashes the caller has marked as visible via M.set_visible_hashes
|
|
-- — otherwise paint stragglers / poll loops for off-screen items would
|
|
-- queue downloads the user can't see.
|
|
function M.trigger_download(hash)
|
|
if _cover_pending[hash] then
|
|
logger.dbg("ReadestLibrary cover skip: " .. tag_for(hash) .. " already in flight")
|
|
return
|
|
end
|
|
if _missing_covers[hash] then
|
|
logger.dbg("ReadestLibrary cover skip: " .. tag_for(hash) .. " known 404")
|
|
return
|
|
end
|
|
if not _opts or not _opts.sync_auth then
|
|
logger.warn("ReadestLibrary cover skip: " .. tag_for(hash)
|
|
.. " — set_opts not called yet")
|
|
return
|
|
end
|
|
if _visible_hashes and not _visible_hashes[hash] then
|
|
logger.dbg("ReadestLibrary cover skip: " .. tag_for(hash) .. " not on visible page")
|
|
return
|
|
end
|
|
|
|
_cover_pending[hash] = true
|
|
table.insert(_download_queue, hash)
|
|
logger.dbg("ReadestLibrary cover queued: " .. tag_for(hash)
|
|
.. " (queue len=" .. #_download_queue .. ")")
|
|
process_queue()
|
|
end
|
|
|
|
-- Set of hashes on the current Menu page; only these may trigger
|
|
-- downloads. Pass nil to disable the filter (e.g. when the Library
|
|
-- closes and the patched BIM might still be invoked from elsewhere).
|
|
function M.set_visible_hashes(menu, cloud_only_flag)
|
|
if not menu then
|
|
logger.dbg("ReadestLibrary set_visible_hashes: cleared (menu nil)")
|
|
_visible_hashes = nil
|
|
return
|
|
end
|
|
local set = {}
|
|
local count = 0
|
|
local page = menu.page or 1
|
|
local perpage = menu.perpage or 1
|
|
local items = menu.item_table or {}
|
|
local first = (page - 1) * perpage + 1
|
|
local last = math.min(first + perpage - 1, #items)
|
|
for i = first, last do
|
|
local entry = items[i]
|
|
if entry and entry[cloud_only_flag] and type(entry.file) == "string" then
|
|
local hash = M.hash_from_uri(entry.file)
|
|
set[hash] = true
|
|
count = count + 1
|
|
end
|
|
end
|
|
_visible_hashes = set
|
|
logger.info("ReadestLibrary set_visible_hashes: page=" .. page
|
|
.. " range=" .. first .. ".." .. last
|
|
.. " cloud_only_visible=" .. count
|
|
.. " (item_table size=" .. #items .. ")")
|
|
end
|
|
|
|
return M
|