ed8956e9ed
* feat(koplugin/library): data layer + busted harness + design doc - LibraryStore: per-user SQLite index merging cloud + local books by partial-md5 hash. listBooks/listBookshelfBooks/listBookshelfGroups, upsertBook with cloud_present/local_present OR-merge + _force/clear sentinels, parseSyncRow, getChangedBooks for tombstone push. - EXTS table mirroring web's document.ts. - busted harness with KOReader stubs (G_reader_settings, DataStorage, lua-ljsqlite3 against :memory:); spec_helper, exts_spec, smoke_spec, librarystore_spec covering schema, sort, group nesting, dedupe. - Library design doc + spec README. - pnpm test:lua wired through root + app package.json; lint-koplugin recurses into library/ + spec/. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(koplugin/library): cloud sync — push, pull, upload, delete, downloads - Spore methods: pullBooks (incremental /sync), getDownloadUrl, getUploadUrl, listFiles, deleteFile. - syncbooks.lua: pushBook + pushChangedBooks (advances watermark to max(updated_at, deleted_at)), syncBooks(opts, mode) for push/pull/both, downloadBook + downloadCover (sync socket.http with file sink; cover download via fork+poll with single-slot queue + visible-page filter + coalesced refresh), uploadBook (presigned-PUT flow + best-effort cover), deleteCloudFiles (list-then-delete-each, mirrors cloudService.deleteBook). - SyncAuth.withFreshToken wrapper resolves the ensureClient race; 401/403 unified across syncconfig + syncannotations. - Cloud + local book covers shared by partial-md5 hash; cover.png cached at <settings>/readest_covers/<hash>.png with sentinel for known 404s. - syncbooks_spec covers row-to-wire conversion + file_key shape. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(koplugin/library): local discovery — sidecar scanner + cover provider - localscanner.lightScan iterates ReadHistory entries, reads partial_md5_checksum from .sdr sidecars, and upserts local rows. Slow filesystem walks deferred to fullSidecarWalk (24h-gated). - coverprovider wraps BookInfoManager:getBookInfo for local books with graceful FakeCover fallback when coverbrowser is absent. - localscanner_spec + coverprovider_spec. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(koplugin/library): UI — widget, item, view menu, FileManager hooks - librarywidget: full-screen Menu mixed in with CoverMenu + Mosaic/List per zen_ui's group_view pattern. Title-bar tap → view menu, search via left icon, drill-in/back for grouping. Async cloud sync deferred via scheduleIn so the menu paints before HTTP fires. - libraryitem: BIM patch for cloud-only (readest-cloud://) and group (readest-group://) URIs; group-cover composer (2x2 mosaic for grid, same in list) with cache key derived from the actual first-N hashes for content-based invalidation; ListMenuItem update + paintTo patches for wider list-mode cover strip and cloud-up/down icon overlay. - libraryviewmenu: ButtonDialog with View/Group by/Sort by/Actions. Default Group by = Groups (parity with web), values authors/groups. - librarypaint: partial-page e-ink repaint shim adapted from zen_ui. - main.lua: Library menu entry, dispatcher actions (Open Library / Push / Pull as general; progress + annotations as reader-only), "Add to Readest" button in FileManager's long-press file dialog (dedupe by partial_md5; bumps updated_at when present, inserts a fresh local-only row otherwise; un-tombstones via _clear_fields). Push books on Library open when auto_sync is on, pull-only otherwise. - Long-press action sheet with Readest BookDetailView parity: Remove from Cloud & Device / Cloud Only / Device Only, Upload to Cloud, Download Book / Cover / All. - Cloud-down + cloud-up SVG icons (LiaCloudDownloadAltSolid / LiaCloudUploadAltSolid) painted in the right-side wpageinfo slot. - i18n catalog updated for new strings. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(koplugin/library): split libraryitem into focused modules libraryitem.lua had grown to 1018 lines mixing five unrelated concerns. Split along the natural seams: cloud_covers — readest-cloud:// URI scheme, on-disk <hash>.png cache, single-slot async download queue, visible-page filter group_covers — readest-group:// URI scheme, 2x2 mosaic composer with content-derived cache key (first-N hashes), cell layout table cloud_icons — bundled cloud-up/cloud-down SVG loader, IconWidget cache, paint-overlay positioning list_strip — list-mode group row builder (4-cover wider strip replacing ListMenu's square cover slot) bim_patch — BookInfoManager:getBookInfo router (cloud / group / local) + ListMenuItem update + paintTo patches; owns the _library_local_paths set and orig BIM reference libraryitem.lua is now 141 lines: just the entry-table constructors (entry_from_row, entry_from_group, entry_back) plus thin install / set_visible_hashes delegates. Each new module is 88-216 lines. No behavior change — same 113 specs pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(koplugin): co-locate dev tooling, ship-zip exclusions, CI job split - apps/readest.koplugin/scripts/build-koplugin.mjs: local sideloading build with the same exclusions the release workflow uses. - Move lint-koplugin + test-koplugin from apps/readest-app/scripts/ to apps/readest.koplugin/scripts/. All koplugin dev tooling now lives with the koplugin and is excluded from the published release zip. - Rename to .mjs so Node treats them as ESM without the reparse warning (the i18n CommonJS scripts stay .js). - Release workflow: zip -r exclusions for scripts/, docs/, spec/, .busted so dev artifacts don't ship to end users. - PR workflow: split build_web_app into build_web_app + test_web_app for parallelism. The test job installs luarocks + busted + lsqlite3complete and runs pnpm test:lua. test-koplugin.mjs now hard-fails (instead of soft-skipping) when CI=true and a tool is missing — a broken CI toolchain previously exited 0 silently. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
163 lines
6.2 KiB
Lua
163 lines
6.2 KiB
Lua
local UIManager = require("ui/uimanager")
|
|
local logger = require("logger")
|
|
local socketutil = require("socketutil")
|
|
|
|
-- Sync operation timeouts
|
|
local SYNC_TIMEOUTS = { 5, 10 }
|
|
|
|
local ReadestSyncClient = {
|
|
service_spec = nil,
|
|
access_token = nil,
|
|
}
|
|
|
|
function ReadestSyncClient:new(o)
|
|
if o == nil then o = {} end
|
|
setmetatable(o, self)
|
|
self.__index = self
|
|
if o.init then o:init() end
|
|
return o
|
|
end
|
|
|
|
function ReadestSyncClient:init()
|
|
local Spore = require("Spore")
|
|
self.client = Spore.new_from_spec(self.service_spec)
|
|
|
|
-- Readest API headers middleware
|
|
package.loaded["Spore.Middleware.ReadestHeaders"] = {}
|
|
require("Spore.Middleware.ReadestHeaders").call = function(args, req)
|
|
req.headers["content-type"] = "application/json"
|
|
req.headers["accept"] = "application/json"
|
|
end
|
|
|
|
-- Readest Bearer token auth middleware
|
|
package.loaded["Spore.Middleware.ReadestAuth"] = {}
|
|
require("Spore.Middleware.ReadestAuth").call = function(args, req)
|
|
if self.access_token then
|
|
req.headers["authorization"] = "Bearer " .. self.access_token
|
|
else
|
|
logger.err("ReadestSyncClient:access_token is not set, cannot authenticate")
|
|
return false, "Access token is required for Readest API"
|
|
end
|
|
end
|
|
|
|
package.loaded["Spore.Middleware.AsyncHTTP"] = {}
|
|
require("Spore.Middleware.AsyncHTTP").call = function(args, req)
|
|
-- disable async http if Turbo looper is missing
|
|
if not UIManager.looper then return end
|
|
req:finalize()
|
|
local result
|
|
require("httpclient"):new():request({
|
|
url = req.url,
|
|
method = req.method,
|
|
body = req.env.spore.payload,
|
|
on_headers = function(headers)
|
|
for header, value in pairs(req.headers) do
|
|
if type(header) == "string" then
|
|
headers:add(header, value)
|
|
end
|
|
end
|
|
end,
|
|
}, function(res)
|
|
result = res
|
|
-- Turbo HTTP client uses code instead of status
|
|
-- change to status so that Spore can understand
|
|
result.status = res.code
|
|
coroutine.resume(args.thread)
|
|
end)
|
|
return coroutine.create(function() coroutine.yield(result) end)
|
|
end
|
|
end
|
|
|
|
-- Internal: prepare the Spore client with our standard middleware stack.
|
|
-- Call before each RPC.
|
|
function ReadestSyncClient:_prepare()
|
|
self.client:reset_middlewares()
|
|
self.client:enable("Format.JSON")
|
|
self.client:enable("ReadestHeaders", {})
|
|
self.client:enable("ReadestAuth", {})
|
|
end
|
|
|
|
-- Internal: dispatch a Spore RPC and invoke `callback(success, body, status)`
|
|
-- when the async response arrives. The status is forwarded so callers can
|
|
-- distinguish 401/403/404 from generic failure (codex round 1 finding 15).
|
|
function ReadestSyncClient:_dispatch(name, args, callback)
|
|
self:_prepare()
|
|
socketutil:set_timeout(SYNC_TIMEOUTS[1], SYNC_TIMEOUTS[2])
|
|
local co = coroutine.create(function()
|
|
local ok, res = pcall(function()
|
|
return self.client[name](self.client, args)
|
|
end)
|
|
if ok then
|
|
callback(res.status == 200, res.body, res.status)
|
|
else
|
|
logger.dbg("ReadestSyncClient:" .. name .. " failure:", res)
|
|
callback(false, res and res.body or nil, res and res.status or nil)
|
|
end
|
|
end)
|
|
self.client:enable("AsyncHTTP", {thread = co})
|
|
coroutine.resume(co)
|
|
if UIManager.looper then UIManager:setInputTimeout() end
|
|
socketutil:reset_timeout()
|
|
end
|
|
|
|
function ReadestSyncClient:pullChanges(params, callback)
|
|
self:_dispatch("pullChanges", {
|
|
since = params.since,
|
|
type = params.type,
|
|
book = params.book,
|
|
meta_hash = params.meta_hash,
|
|
}, callback)
|
|
end
|
|
|
|
function ReadestSyncClient:pushChanges(changes, callback)
|
|
self:_dispatch("pushChanges", changes or {}, callback)
|
|
end
|
|
|
|
-- pullBooks: incremental fetch of the books table since the watermark.
|
|
-- Returns body shape `{ books: [...] }`. Drives Library:open() refresh.
|
|
function ReadestSyncClient:pullBooks(params, callback)
|
|
self:_dispatch("pullBooks", { since = params.since }, callback)
|
|
end
|
|
|
|
-- getDownloadUrl: resolve a storage fileKey to a signed URL. The server's
|
|
-- processFileKeys fallback at apps/readest-app/src/pages/api/storage/
|
|
-- download.ts:99-107 lets us send the simple {hash}/{hash}.{ext} variant
|
|
-- and have R2 deployments resolve to the actual stored filename
|
|
-- transparently. Body shape on success: { downloadUrl }.
|
|
function ReadestSyncClient:getDownloadUrl(params, callback)
|
|
self:_dispatch("getDownloadUrl", { fileKey = params.fileKey }, callback)
|
|
end
|
|
|
|
-- listFiles: enumerate the rows of the `files` table for a given book
|
|
-- hash. Used to discover the EXACT fileKeys (book object + cover.png)
|
|
-- before deletion — the DELETE endpoint requires a literal match (no
|
|
-- extension fallback like /storage/download has). Body shape on success:
|
|
-- { files: [ { file_key, file_size, book_hash, ... } ] }.
|
|
function ReadestSyncClient:listFiles(params, callback)
|
|
self:_dispatch("listFiles", { bookHash = params.bookHash }, callback)
|
|
end
|
|
|
|
-- deleteFile: remove one storage object plus its `files` row. Caller
|
|
-- usually iterates over listFiles output. The `books` row stays put;
|
|
-- cleanup of the books table is a separate /sync push (with deletedAt).
|
|
function ReadestSyncClient:deleteFile(params, callback)
|
|
self:_dispatch("deleteFile", { fileKey = params.fileKey }, callback)
|
|
end
|
|
|
|
-- getUploadUrl: ask the server to issue a presigned PUT URL for a new
|
|
-- storage object. Two-step flow (`apps/readest-app/src/pages/api/storage/
|
|
-- upload.ts`): server inserts a row in the `files` table for
|
|
-- (user, bookHash, fileKey) BEFORE the actual bytes move. Body shape on
|
|
-- success: { uploadUrl, fileKey, usage, quota }. Quota-exceeded → 403.
|
|
-- The fileName must be the cloud-relative path (eg
|
|
-- "Readest/Books/<hash>/<hash>.epub"); the server prepends "<user.id>/"
|
|
-- to form the final fileKey.
|
|
function ReadestSyncClient:getUploadUrl(params, callback)
|
|
self:_dispatch("getUploadUrl", {
|
|
fileName = params.fileName,
|
|
fileSize = params.fileSize,
|
|
bookHash = params.bookHash,
|
|
}, callback)
|
|
end
|
|
|
|
return ReadestSyncClient |