ed8956e9ed
* feat(koplugin/library): data layer + busted harness + design doc - LibraryStore: per-user SQLite index merging cloud + local books by partial-md5 hash. listBooks/listBookshelfBooks/listBookshelfGroups, upsertBook with cloud_present/local_present OR-merge + _force/clear sentinels, parseSyncRow, getChangedBooks for tombstone push. - EXTS table mirroring web's document.ts. - busted harness with KOReader stubs (G_reader_settings, DataStorage, lua-ljsqlite3 against :memory:); spec_helper, exts_spec, smoke_spec, librarystore_spec covering schema, sort, group nesting, dedupe. - Library design doc + spec README. - pnpm test:lua wired through root + app package.json; lint-koplugin recurses into library/ + spec/. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(koplugin/library): cloud sync — push, pull, upload, delete, downloads - Spore methods: pullBooks (incremental /sync), getDownloadUrl, getUploadUrl, listFiles, deleteFile. - syncbooks.lua: pushBook + pushChangedBooks (advances watermark to max(updated_at, deleted_at)), syncBooks(opts, mode) for push/pull/both, downloadBook + downloadCover (sync socket.http with file sink; cover download via fork+poll with single-slot queue + visible-page filter + coalesced refresh), uploadBook (presigned-PUT flow + best-effort cover), deleteCloudFiles (list-then-delete-each, mirrors cloudService.deleteBook). - SyncAuth.withFreshToken wrapper resolves the ensureClient race; 401/403 unified across syncconfig + syncannotations. - Cloud + local book covers shared by partial-md5 hash; cover.png cached at <settings>/readest_covers/<hash>.png with sentinel for known 404s. - syncbooks_spec covers row-to-wire conversion + file_key shape. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(koplugin/library): local discovery — sidecar scanner + cover provider - localscanner.lightScan iterates ReadHistory entries, reads partial_md5_checksum from .sdr sidecars, and upserts local rows. Slow filesystem walks deferred to fullSidecarWalk (24h-gated). - coverprovider wraps BookInfoManager:getBookInfo for local books with graceful FakeCover fallback when coverbrowser is absent. - localscanner_spec + coverprovider_spec. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(koplugin/library): UI — widget, item, view menu, FileManager hooks - librarywidget: full-screen Menu mixed in with CoverMenu + Mosaic/List per zen_ui's group_view pattern. Title-bar tap → view menu, search via left icon, drill-in/back for grouping. Async cloud sync deferred via scheduleIn so the menu paints before HTTP fires. - libraryitem: BIM patch for cloud-only (readest-cloud://) and group (readest-group://) URIs; group-cover composer (2x2 mosaic for grid, same in list) with cache key derived from the actual first-N hashes for content-based invalidation; ListMenuItem update + paintTo patches for wider list-mode cover strip and cloud-up/down icon overlay. - libraryviewmenu: ButtonDialog with View/Group by/Sort by/Actions. Default Group by = Groups (parity with web), values authors/groups. - librarypaint: partial-page e-ink repaint shim adapted from zen_ui. - main.lua: Library menu entry, dispatcher actions (Open Library / Push / Pull as general; progress + annotations as reader-only), "Add to Readest" button in FileManager's long-press file dialog (dedupe by partial_md5; bumps updated_at when present, inserts a fresh local-only row otherwise; un-tombstones via _clear_fields). Push books on Library open when auto_sync is on, pull-only otherwise. - Long-press action sheet with Readest BookDetailView parity: Remove from Cloud & Device / Cloud Only / Device Only, Upload to Cloud, Download Book / Cover / All. - Cloud-down + cloud-up SVG icons (LiaCloudDownloadAltSolid / LiaCloudUploadAltSolid) painted in the right-side wpageinfo slot. - i18n catalog updated for new strings. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(koplugin/library): split libraryitem into focused modules libraryitem.lua had grown to 1018 lines mixing five unrelated concerns. Split along the natural seams: cloud_covers — readest-cloud:// URI scheme, on-disk <hash>.png cache, single-slot async download queue, visible-page filter group_covers — readest-group:// URI scheme, 2x2 mosaic composer with content-derived cache key (first-N hashes), cell layout table cloud_icons — bundled cloud-up/cloud-down SVG loader, IconWidget cache, paint-overlay positioning list_strip — list-mode group row builder (4-cover wider strip replacing ListMenu's square cover slot) bim_patch — BookInfoManager:getBookInfo router (cloud / group / local) + ListMenuItem update + paintTo patches; owns the _library_local_paths set and orig BIM reference libraryitem.lua is now 141 lines: just the entry-table constructors (entry_from_row, entry_from_group, entry_back) plus thin install / set_visible_hashes delegates. Each new module is 88-216 lines. No behavior change — same 113 specs pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(koplugin): co-locate dev tooling, ship-zip exclusions, CI job split - apps/readest.koplugin/scripts/build-koplugin.mjs: local sideloading build with the same exclusions the release workflow uses. - Move lint-koplugin + test-koplugin from apps/readest-app/scripts/ to apps/readest.koplugin/scripts/. All koplugin dev tooling now lives with the koplugin and is excluded from the published release zip. - Rename to .mjs so Node treats them as ESM without the reparse warning (the i18n CommonJS scripts stay .js). - Release workflow: zip -r exclusions for scripts/, docs/, spec/, .busted so dev artifacts don't ship to end users. - PR workflow: split build_web_app into build_web_app + test_web_app for parallelism. The test job installs luarocks + busted + lsqlite3complete and runs pnpm test:lua. test-koplugin.mjs now hard-fails (instead of soft-skipping) when CI=true and a tool is missing — a broken CI toolchain previously exited 0 silently. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
295 lines
9.6 KiB
Lua
295 lines
9.6 KiB
Lua
local Event = require("ui/event")
|
|
local InfoMessage = require("ui/widget/infomessage")
|
|
local UIManager = require("ui/uimanager")
|
|
local logger = require("logger")
|
|
local util = require("util")
|
|
local sha2 = require("ffi/sha2")
|
|
local _ = require("i18n")
|
|
|
|
local SyncConfig = {}
|
|
|
|
local function normalizeIdentifier(identifier)
|
|
if identifier:match("urn:") then
|
|
return identifier:match("([^:]+)$")
|
|
elseif identifier:match(":") then
|
|
return identifier:match("^[^:]+:(.+)$")
|
|
end
|
|
return identifier
|
|
end
|
|
|
|
local function normalizeAuthor(author)
|
|
author = author:gsub("^%s*(.-)%s*$", "%1")
|
|
return author
|
|
end
|
|
|
|
function SyncConfig:getMetadataHashInfo(ui)
|
|
local doc_props = ui.doc_settings:readSetting("doc_props") or {}
|
|
local title = doc_props.title or ''
|
|
if title == '' then
|
|
local _doc_path, filename = util.splitFilePathName(ui.doc_settings:readSetting("doc_path") or '')
|
|
local basename, _suffix = util.splitFileNameSuffix(filename)
|
|
title = basename or ''
|
|
end
|
|
|
|
local authors_raw = doc_props.authors or ''
|
|
local authors_list = {}
|
|
if authors_raw:find("\n") then
|
|
local list = util.splitToArray(authors_raw, "\n")
|
|
for i, author in ipairs(list) do
|
|
authors_list[i] = normalizeAuthor(author)
|
|
end
|
|
elseif authors_raw ~= '' then
|
|
authors_list = { normalizeAuthor(authors_raw) }
|
|
end
|
|
|
|
local identifiers_raw = doc_props.identifiers or ''
|
|
local identifiers_list = {}
|
|
if identifiers_raw:find("\n") then
|
|
local list = util.splitToArray(identifiers_raw, "\n")
|
|
local normalized = {}
|
|
local priorities = { "uuid", "calibre", "isbn" }
|
|
local preferred = nil
|
|
for i, id in ipairs(list) do
|
|
normalized[i] = normalizeIdentifier(id)
|
|
local candidate = id:lower()
|
|
for _, p in ipairs(priorities) do
|
|
if candidate:find(p, 1, true) then
|
|
preferred = normalized[i]
|
|
break
|
|
end
|
|
end
|
|
end
|
|
if preferred then
|
|
identifiers_list = { preferred }
|
|
else
|
|
identifiers_list = normalized
|
|
end
|
|
elseif identifiers_raw ~= '' then
|
|
identifiers_list = { normalizeIdentifier(identifiers_raw) }
|
|
end
|
|
|
|
local hash_source = title .. "|" .. table.concat(authors_list, ",") .. "|" .. table.concat(identifiers_list, ",")
|
|
return {
|
|
title = title,
|
|
authors = authors_list,
|
|
identifiers = identifiers_list,
|
|
hash_source = hash_source,
|
|
meta_hash = sha2.md5(hash_source),
|
|
}
|
|
end
|
|
|
|
function SyncConfig:generateMetadataHash(ui)
|
|
return self:getMetadataHashInfo(ui).meta_hash
|
|
end
|
|
|
|
function SyncConfig:getMetaHash(ui)
|
|
local doc_readest_sync = ui.doc_settings:readSetting("readest_sync") or {}
|
|
local meta_hash = doc_readest_sync.meta_hash_v1
|
|
if not meta_hash then
|
|
meta_hash = self:generateMetadataHash(ui)
|
|
doc_readest_sync.meta_hash_v1 = meta_hash
|
|
ui.doc_settings:saveSetting("readest_sync", doc_readest_sync)
|
|
end
|
|
return meta_hash
|
|
end
|
|
|
|
function SyncConfig:getDocumentIdentifier(ui)
|
|
return ui.doc_settings:readSetting("partial_md5_checksum")
|
|
end
|
|
|
|
function SyncConfig:getCurrentBookConfig(ui)
|
|
local book_hash = self:getDocumentIdentifier(ui)
|
|
local meta_hash = self:getMetaHash(ui)
|
|
if not book_hash or not meta_hash then
|
|
UIManager:show(InfoMessage:new{
|
|
text = _("Cannot identify the current book"),
|
|
timeout = 2,
|
|
})
|
|
return nil
|
|
end
|
|
|
|
local config = {
|
|
bookHash = book_hash,
|
|
metaHash = meta_hash,
|
|
progress = "",
|
|
xpointer = "",
|
|
updatedAt = os.time() * 1000,
|
|
}
|
|
|
|
local current_page = ui:getCurrentPage()
|
|
local page_count = ui.document:getPageCount()
|
|
config.progress = {current_page, page_count}
|
|
|
|
if not ui.document.info.has_pages then
|
|
config.xpointer = ui.rolling:getLastProgress()
|
|
end
|
|
|
|
return config
|
|
end
|
|
|
|
function SyncConfig:applyBookConfig(ui, config)
|
|
logger.dbg("ReadestSync: Applying book config:", config)
|
|
local xpointer = config.xpointer
|
|
local progress = config.progress
|
|
local has_pages = ui.document.info.has_pages
|
|
local progress_pattern = "^%[(%d+),(%d+)%]$"
|
|
if has_pages and progress then
|
|
local page, _total_pages = progress:match(progress_pattern)
|
|
local current_page = ui:getCurrentPage()
|
|
local new_page = tonumber(page)
|
|
if new_page > current_page then
|
|
ui.link:addCurrentLocationToStack()
|
|
ui:handleEvent(Event:new("GotoPage", new_page))
|
|
self:showSyncedMessage()
|
|
end
|
|
end
|
|
if not has_pages and xpointer then
|
|
local last_xpointer = ui.rolling:getLastProgress()
|
|
local working_xpointer = xpointer
|
|
local cmp_result = ui.document:compareXPointers(last_xpointer, working_xpointer)
|
|
while cmp_result == nil and working_xpointer do
|
|
local last_slash_pos = working_xpointer:match("^.*()/")
|
|
if last_slash_pos and last_slash_pos > 1 then
|
|
working_xpointer = working_xpointer:sub(1, last_slash_pos - 1)
|
|
cmp_result = ui.document:compareXPointers(last_xpointer, working_xpointer)
|
|
else
|
|
break
|
|
end
|
|
end
|
|
if cmp_result > 0 then
|
|
ui.link:addCurrentLocationToStack()
|
|
ui:handleEvent(Event:new("GotoXPointer", working_xpointer))
|
|
self:showSyncedMessage()
|
|
end
|
|
end
|
|
end
|
|
|
|
function SyncConfig:showSyncedMessage()
|
|
UIManager:show(InfoMessage:new{
|
|
text = _("Progress has been synchronized."),
|
|
timeout = 3,
|
|
})
|
|
end
|
|
|
|
function SyncConfig:push(ui, settings, client, interactive, last_sync_timestamp)
|
|
local config = self:getCurrentBookConfig(ui)
|
|
if not config then return last_sync_timestamp end
|
|
|
|
if interactive then
|
|
UIManager:show(InfoMessage:new{
|
|
text = _("Pushing reading progress..."),
|
|
timeout = 1,
|
|
})
|
|
end
|
|
|
|
local payload = {
|
|
books = {},
|
|
notes = {},
|
|
configs = { config },
|
|
}
|
|
|
|
client:pushChanges(
|
|
payload,
|
|
function(success, _response)
|
|
if interactive then
|
|
if success then
|
|
UIManager:show(InfoMessage:new{
|
|
text = _("Reading progress pushed successfully"),
|
|
timeout = 2,
|
|
})
|
|
else
|
|
UIManager:show(InfoMessage:new{
|
|
text = _("Failed to push reading progress"),
|
|
timeout = 2,
|
|
})
|
|
end
|
|
end
|
|
if success and ui.doc_settings then
|
|
local doc_readest_sync = ui.doc_settings:readSetting("readest_sync") or {}
|
|
doc_readest_sync.last_synced_at_config = os.time()
|
|
ui.doc_settings:saveSetting("readest_sync", doc_readest_sync)
|
|
end
|
|
end
|
|
)
|
|
|
|
if not interactive then
|
|
return os.time()
|
|
end
|
|
return last_sync_timestamp
|
|
end
|
|
|
|
function SyncConfig:pull(ui, settings, client, book_hash, meta_hash, interactive, logout_fn)
|
|
if interactive then
|
|
UIManager:show(InfoMessage:new{
|
|
text = _("Pulling reading progress..."),
|
|
timeout = 1,
|
|
})
|
|
end
|
|
|
|
client:pullChanges(
|
|
{
|
|
since = 0,
|
|
type = "configs",
|
|
book = book_hash,
|
|
meta_hash = meta_hash,
|
|
},
|
|
function(success, response, status)
|
|
if not success then
|
|
-- Auth failure: server returns HTTP 403 with body
|
|
-- {error="Not authenticated"} per apps/readest-app/src/pages/api/sync.ts:31.
|
|
-- Check the status code primarily so future endpoints with
|
|
-- different body shapes still trigger relogin (codex finding).
|
|
local is_auth_fail = status == 401 or status == 403
|
|
or (response and response.error == "Not authenticated")
|
|
if is_auth_fail then
|
|
if interactive then
|
|
UIManager:show(InfoMessage:new{
|
|
text = _("Authentication failed, please login again"),
|
|
timeout = 2,
|
|
})
|
|
end
|
|
if logout_fn then logout_fn() end
|
|
return
|
|
end
|
|
if interactive then
|
|
UIManager:show(InfoMessage:new{
|
|
text = _("Failed to pull reading progress"),
|
|
timeout = 2,
|
|
})
|
|
end
|
|
return
|
|
end
|
|
|
|
if ui.doc_settings then
|
|
local doc_readest_sync = ui.doc_settings:readSetting("readest_sync") or {}
|
|
doc_readest_sync.last_synced_at_config = os.time()
|
|
ui.doc_settings:saveSetting("readest_sync", doc_readest_sync)
|
|
end
|
|
|
|
local data = response.configs
|
|
if data and #data > 0 then
|
|
local config = data[1]
|
|
if config then
|
|
self:applyBookConfig(ui, config)
|
|
if interactive then
|
|
UIManager:show(InfoMessage:new{
|
|
text = _("Reading progress synchronized"),
|
|
timeout = 2,
|
|
})
|
|
end
|
|
return
|
|
end
|
|
end
|
|
|
|
if interactive then
|
|
UIManager:show(InfoMessage:new{
|
|
text = _("No saved reading progress found for this book"),
|
|
timeout = 2,
|
|
})
|
|
end
|
|
end
|
|
)
|
|
end
|
|
|
|
return SyncConfig
|