* ci(pull-request): cache the vendored tauri workspace crates
build_tauri_app rebuilt the whole tauri stack every run. The fork is wired via
[patch.crates-io] to path crates (packages/tauri, packages/tauri-plugins) plus
local src-tauri/plugins/*, all workspace members that Swatinem/rust-cache prunes
by default (cache-workspace-crates: false). Every crates.io plugin depending on
the patched `tauri` then rebuilt transitively, while unrelated deps stayed cached.
Set cache-workspace-crates: true so those sporadically-updated submodule crates
are cached, and bump the cache key (tauri-cargo -> tauri-cargo-ws) so the old
workspace-crate-less cache is invalidated and repopulated (rust-cache won't
re-save on a full key match). The first run after this is a full rebuild;
subsequent runs reuse the cached tauri stack.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* ci(pull-request): keep a single rust-cache for build_tauri_app
build_tauri_app ran two rust-cache actions: actions-rust-lang/setup-rust-toolchain's
built-in one (cache-workspace-crates: false) plus the explicit Swatinem/rust-cache.
They doubled cache storage and competed over the shared target/. Set cache: false on
setup-rust-toolchain so the explicit cache — the one configured with
cache-workspace-crates for the vendored tauri fork — is the only one.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* ci(security): pin android-emulator-runner action by commit SHA
Scorecard Pinned-Dependencies flagged the two
reactivecircus/android-emulator-runner@v2 usages in android-e2e.yml as
third-party actions not pinned by hash (code-scanning alerts #116, #117).
Pin both to the full commit SHA the v2 tag currently resolves to
(e89f39f = v2.37.0), matching the @<sha> # <version> convention already
used by every other action in this workflow.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* ci(pull-request): shard web unit tests and split out test_extensions
The jsdom unit suite was ~115s of the 280s test_web_app job — the slowest
check on every PR. Split it across two parallel shards with vitest --shard,
and move the browser-extension + koplugin tests into a new test_extensions
job.
- test_web_app: matrix shard [1, 2] running `vitest run --shard=N/2`;
Playwright + browser tests run on shard 1 only.
- test_extensions: extension tests + browser-ext build run always; the
koplugin Lua tests (and their ~45s LuaJIT/busted install) run only when
apps/readest.koplugin/** changed, detected via dorny/paths-filter
(pinned by SHA; needs pull-requests: read to list PR files).
- package.json: add test:pr:web:unit so CI can append --shard;
test:pr:web still runs the full sequence locally.
Cuts the PR critical path from ~280s toward ~188s (now build_tauri_app).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* ci(pull-request): isolate koplugin lint + LuaJIT install into test_extensions
build_web_app was the slowest PR job and installed LuaJIT + ran the koplugin
syntax check on every PR. Move all koplugin tooling into test_extensions,
gated on apps/readest.koplugin/** like the koplugin Lua tests already are:
- build_web_app drops the LuaJIT install.
- test_extensions installs LuaJIT/busted and runs `pnpm lint:lua` + `pnpm
test:lua` only when the koplugin sources changed.
- `pnpm lint` is now web-only (tsgo + biome); `lint:lua` stays a standalone
script that test_extensions (and local koplugin work) calls directly. This
also drops koplugin lint from the pre-push hook.
- verification rule updated to match.
Most PRs now skip the koplugin toolchain entirely.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(reader): random-access file reads on Android via rangefile scheme
NativeFile's per-chunk Tauri IPC (open+seek+read+close) is slow on Android, and RemoteFile can't replace it because the WebView mishandles Range requests on intercepted custom-protocol responses — it re-applies the offset to the already-sliced body, so any non-zero-start range returns corrupt data or net::ERR_FAILED (Chromium 40739128, tauri-apps/tauri#12019/#3725).
Add a `rangefile` custom URI scheme that carries the byte range in the URL query (?path=&start=&end=) instead of a Range header. With no Range header the WebView delivers the 200 body verbatim, while bytes still stream through the network stack rather than the IPC bridge. The handler is scope-gated by asset_protocol_scope (same boundary as the asset protocol) plus an explicit traversal/NUL/relative guard.
RemoteFile.fromNativePath() drives the scheme on Android (query-carried range, X-Total-Size for size); nativeAppService.openFile routes Android reads through it with a NativeFile fallback. Verified on-device (Android 16 / WebView 147) via CDP: byte-equal reads at every offset, ~1.8x faster small scattered reads, real book opens/renders; all out-of-scope/traversal/NUL paths rejected 403.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* ci(rust): run cargo unit tests in rust_lint
The rust_lint job ran only fmt + clippy, so the crate's ~40 Rust unit tests (parsers, parser_common, and the new range_file tests) never executed in CI. Add `cargo test -p Readest --lib` to rust_lint — the frontend dist is absent there, but generate_context! already compiles without it (clippy proves this) and the unit tests run headless.
Also add a `test:rust` pnpm script and document it as verification done-condition #6.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* ci: optimize build time for Docker and CI workflows
- Dockerfile: slim production-stage to copy only runtime artifacts
(.next, public, node_modules, package.json, next.config.mjs),
dropping src/, src-tauri/, patches/, packages source, etc.
- Dockerfile: add sharing=locked to pnpm store cache mount to prevent
concurrent-build cache corruption
- docker-image.yml: pin actions/checkout to SHA (consistent with other workflows)
- docker-image.yml: switch Buildx cache from type=gha (10 GB shared limit,
poor for multi-arch) to type=registry on GHCR (no size cap, already
authenticated, correct per-platform caching)
- pull-request.yml: use pnpm install --frozen-lockfile --prefer-offline
- release.yml: use pnpm install --frozen-lockfile --prefer-offline
- next.config.mjs: skip redundant eslint pass during next build
(lint already runs as a dedicated CI step)
* fix(docker): eliminate QEMU emulation, fix pnpm version mismatch, patch artifact write CVE (#4)
* fix(docker): eliminate QEMU arm64 emulation and fix pnpm version mismatch
- Fix pnpm version in Dockerfile: 10.29.3 → 11.1.1 (matches package.json)
Prevents corepack from re-downloading pnpm 11 on every build
- Replace single QEMU job with matrix build (ubuntu-latest for amd64,
ubuntu-24.04-arm for arm64) — eliminates ~21 min QEMU emulation overhead
- Use per-platform build cache tags (buildcache-linux-amd64 / buildcache-linux-arm64)
to avoid cache thrashing between architectures
- Add merge job that assembles multi-arch manifest from platform digests
Agent-Logs-Url: https://github.com/pourmand1376/readest/sessions/89c298b4-8a58-4517-ac7f-2f26c86bbcd6
Co-authored-by: pourmand1376 <32064808+pourmand1376@users.noreply.github.com>
* fix(ci): upgrade artifact actions to v7 to patch arbitrary file write vulnerability
actions/download-artifact >= 4.0.0 < 4.1.3 allows arbitrary file write
via artifact extraction. Pin both upload-artifact and download-artifact to
v7 (SHA-pinned), consistent with the rest of the repo's workflows.
Agent-Logs-Url: https://github.com/pourmand1376/readest/sessions/89c298b4-8a58-4517-ac7f-2f26c86bbcd6
Co-authored-by: pourmand1376 <32064808+pourmand1376@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: pourmand1376 <32064808+pourmand1376@users.noreply.github.com>
* chore(docker): tighten build context
Exclude local env, build output, credential, and tooling state files from Docker build context to reduce registry cache exposure.
---------
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Huang Xin <chrox.huang@gmail.com>
* ci(e2e): cache Playwright browsers and apt packages
- cache `~/.cache/ms-playwright` keyed on the lockfile; on a hit only
the OS deps are installed, skipping the browser download
- cache apt archives in test_web_app, matching the rust/tauri jobs
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* ci: enable Turbopack persistent cache and key it for cross-PR reuse
Enable `experimental.turbopackFileSystemCacheForBuild` so `next build`
persists a real Turbopack cache (~640 MB at `.next/cache/turbopack`),
instead of the ~340 KB of metadata the previous `.next/cache` cache
held. Dev caching is already on by default in Next 16.1+.
Redesign the cache keys so they actually pay off:
- drop `${{ github.sha }}` from the key — it made every commit a unique
entry that no other PR could exact-hit. The key is now
`turbo-<mode>-<target>-<os>-<lockfile-hash>`, deterministic across
branches, so every PR restores the same entry (in practice the one
`main` last saved — the only cache sibling PRs can all see).
- `build_web_app` (`next build`) caches `.next/cache`;
`build_tauri_app` (`next dev`) caches `.next/dev/cache` — `next dev`'s
Turbopack cache lives in a different directory.
- drop the Next.js cache step from `test_web_app`; it runs no build.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(e2e): add Playwright web e2e lane
Adds a web-layer end-to-end suite that drives the Next.js web build
(`pnpm dev-web`) in a real browser, complementing the existing
WebdriverIO suite that drives the Tauri shell.
- playwright.config.ts: single Chromium project, auto-starts dev-web
- e2e/pages: BasePage/LibraryPage/ReaderPage page objects
- e2e/fixtures/base.ts: suppresses demo-book auto-import for a
deterministic empty library
- e2e/tests: library shell + search, book import, reader open +
pagination smoke specs
- e2e/fixtures/books: synthetic sample book for import tests
- scripts: test:e2e:web, test:e2e:web:ui, test:e2e:web:report
Tests run unauthenticated against isolated browser contexts;
authenticated/sync flows are out of scope until a test account is
provisioned.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(e2e): cover reading and annotation flows
Expands the Playwright web e2e lane beyond library/import smoke tests to
exercise the major reading and annotation features against the real
sample-alice.epub fixture (src/__tests__/fixtures/data/).
Reading (reading.spec.ts): open + page turn, TOC chapter navigation,
in-book search, font-size change via the settings dialog, bookmark
toggle.
Annotation (annotation.spec.ts): selection popup, create highlight,
change highlight color, add a note, delete an annotation.
- ReaderPage POM gains sidebar/TOC, search, settings, bookmark and
annotation actions; text selection is driven inside the section
iframe (synthetic drags do not produce a selection through nested
paginated foliate iframes)
- openBook fixture imports and opens a book so specs skip boilerplate
- books.ts centralises fixture book paths
- replaces the old reader.spec.ts smoke
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(e2e): add headed run script and always write HTML report
- test:e2e:web:headed runs the suite in a visible browser, one test at
a time, with traces captured
- the HTML reporter now runs for local runs too, so every run writes
playwright-report/ for test:e2e:web:report to open
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(e2e): fix headed-run flakes in reading and annotation specs
The headed run (slower rendering) surfaced two races that the headless
run happened to pass:
- TOC navigation read reading progress before the section's async
progress update landed — now polls with expect.poll.
- visibleSectionFrame required a paragraph fully inside the viewport,
which intermittently matched nothing — now accepts any paragraph
intersecting the viewport and tolerates frames detaching mid-navigation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* ci: run the Playwright web e2e suite in test_web_app
Adds `pnpm test:e2e:web` to the test_web_app job, after the unit/browser
tests. The job already installs the Chromium browser, and `.env.web` is
committed so the auto-started `pnpm dev-web` server has its config. On
failure the HTML report is uploaded as an artifact.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(e2e): exclude e2e specs from the vitest run
vitest's default glob matches `*.spec.ts`, so it picked up the new
Playwright `e2e/tests/*.spec.ts` files and crashed. Exclude `e2e/`
from vitest — those specs run via `pnpm test:e2e:web`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* ci(e2e): run the web e2e suite against a production build
`next dev` renders a full-screen error overlay when the app emits its
`next-view-transitions` "Transition was aborted" unhandled rejection,
and the overlay intercepts pointer events — making the suite flaky on
CI. CI now builds the web app (`pnpm build-web`) and the Playwright
webServer serves it via `pnpm start-web`; local runs still use
`pnpm dev-web`. Verified: 14/14 pass against the production build.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* ci(e2e): run the web e2e suite in the build_web_app job
build_web_app already runs `pnpm build-web`, so the e2e suite belongs
there — it reuses that build (the CI Playwright webServer serves it via
`pnpm start-web`) instead of building a second time in test_web_app.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(e2e): run the web e2e suite with 4 workers
Specs are isolated (a fresh browser context per test), so they are
safe to parallelize. `test:e2e:web:headed` keeps --workers=1.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(koplugin/library): data layer + busted harness + design doc
- LibraryStore: per-user SQLite index merging cloud + local books by
partial-md5 hash. listBooks/listBookshelfBooks/listBookshelfGroups,
upsertBook with cloud_present/local_present OR-merge + _force/clear
sentinels, parseSyncRow, getChangedBooks for tombstone push.
- EXTS table mirroring web's document.ts.
- busted harness with KOReader stubs (G_reader_settings, DataStorage,
lua-ljsqlite3 against :memory:); spec_helper, exts_spec, smoke_spec,
librarystore_spec covering schema, sort, group nesting, dedupe.
- Library design doc + spec README.
- pnpm test:lua wired through root + app package.json; lint-koplugin
recurses into library/ + spec/.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(koplugin/library): cloud sync — push, pull, upload, delete, downloads
- Spore methods: pullBooks (incremental /sync), getDownloadUrl,
getUploadUrl, listFiles, deleteFile.
- syncbooks.lua: pushBook + pushChangedBooks (advances watermark to
max(updated_at, deleted_at)), syncBooks(opts, mode) for push/pull/both,
downloadBook + downloadCover (sync socket.http with file sink; cover
download via fork+poll with single-slot queue + visible-page filter +
coalesced refresh), uploadBook (presigned-PUT flow + best-effort
cover), deleteCloudFiles (list-then-delete-each, mirrors
cloudService.deleteBook).
- SyncAuth.withFreshToken wrapper resolves the ensureClient race; 401/403
unified across syncconfig + syncannotations.
- Cloud + local book covers shared by partial-md5 hash; cover.png cached
at <settings>/readest_covers/<hash>.png with sentinel for known 404s.
- syncbooks_spec covers row-to-wire conversion + file_key shape.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(koplugin/library): local discovery — sidecar scanner + cover provider
- localscanner.lightScan iterates ReadHistory entries, reads
partial_md5_checksum from .sdr sidecars, and upserts local rows.
Slow filesystem walks deferred to fullSidecarWalk (24h-gated).
- coverprovider wraps BookInfoManager:getBookInfo for local books with
graceful FakeCover fallback when coverbrowser is absent.
- localscanner_spec + coverprovider_spec.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(koplugin/library): UI — widget, item, view menu, FileManager hooks
- librarywidget: full-screen Menu mixed in with CoverMenu + Mosaic/List
per zen_ui's group_view pattern. Title-bar tap → view menu, search via
left icon, drill-in/back for grouping. Async cloud sync deferred via
scheduleIn so the menu paints before HTTP fires.
- libraryitem: BIM patch for cloud-only (readest-cloud://) and group
(readest-group://) URIs; group-cover composer (2x2 mosaic for grid,
same in list) with cache key derived from the actual first-N hashes
for content-based invalidation; ListMenuItem update + paintTo patches
for wider list-mode cover strip and cloud-up/down icon overlay.
- libraryviewmenu: ButtonDialog with View/Group by/Sort by/Actions.
Default Group by = Groups (parity with web), values authors/groups.
- librarypaint: partial-page e-ink repaint shim adapted from zen_ui.
- main.lua: Library menu entry, dispatcher actions (Open Library / Push
/ Pull as general; progress + annotations as reader-only),
"Add to Readest" button in FileManager's long-press file dialog
(dedupe by partial_md5; bumps updated_at when present, inserts a
fresh local-only row otherwise; un-tombstones via _clear_fields).
Push books on Library open when auto_sync is on, pull-only otherwise.
- Long-press action sheet with Readest BookDetailView parity:
Remove from Cloud & Device / Cloud Only / Device Only,
Upload to Cloud, Download Book / Cover / All.
- Cloud-down + cloud-up SVG icons (LiaCloudDownloadAltSolid /
LiaCloudUploadAltSolid) painted in the right-side wpageinfo slot.
- i18n catalog updated for new strings.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(koplugin/library): split libraryitem into focused modules
libraryitem.lua had grown to 1018 lines mixing five unrelated
concerns. Split along the natural seams:
cloud_covers — readest-cloud:// URI scheme, on-disk <hash>.png
cache, single-slot async download queue, visible-page
filter
group_covers — readest-group:// URI scheme, 2x2 mosaic composer with
content-derived cache key (first-N hashes), cell
layout table
cloud_icons — bundled cloud-up/cloud-down SVG loader, IconWidget
cache, paint-overlay positioning
list_strip — list-mode group row builder (4-cover wider strip
replacing ListMenu's square cover slot)
bim_patch — BookInfoManager:getBookInfo router (cloud / group /
local) + ListMenuItem update + paintTo patches; owns
the _library_local_paths set and orig BIM reference
libraryitem.lua is now 141 lines: just the entry-table constructors
(entry_from_row, entry_from_group, entry_back) plus thin install /
set_visible_hashes delegates. Each new module is 88-216 lines.
No behavior change — same 113 specs pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore(koplugin): co-locate dev tooling, ship-zip exclusions, CI job split
- apps/readest.koplugin/scripts/build-koplugin.mjs: local sideloading
build with the same exclusions the release workflow uses.
- Move lint-koplugin + test-koplugin from apps/readest-app/scripts/ to
apps/readest.koplugin/scripts/. All koplugin dev tooling now lives
with the koplugin and is excluded from the published release zip.
- Rename to .mjs so Node treats them as ESM without the reparse warning
(the i18n CommonJS scripts stay .js).
- Release workflow: zip -r exclusions for scripts/, docs/, spec/,
.busted so dev artifacts don't ship to end users.
- PR workflow: split build_web_app into build_web_app + test_web_app
for parallelism. The test job installs luarocks + busted +
lsqlite3complete and runs pnpm test:lua. test-koplugin.mjs now
hard-fails (instead of soft-skipping) when CI=true and a tool is
missing — a broken CI toolchain previously exited 0 silently.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- i18n loader at apps/readest.koplugin/i18n.lua: isolated callable,
falls back to KOReader's gettext when a string is untranslated
- Translation catalog at locales/<i18next-code>/translation.po for 31
languages, mirroring apps/readest-app/public/locales/
- scripts/extract-i18n.js (scan _("...") and _([[...]]); preserve
existing, drop obsolete, add new) and scripts/apply-translations.js
(bulk import from /tmp/koplugin-translations/<lang>.json)
- Mirror apps/readest-app SyncInfoDialog: rename showMetaHashInfo to
showSyncInfo, dialog title "Sync Info", new Last Synced row computed
as max(last_synced_at_config, last_synced_at_notes) from doc_settings
- syncconfig.lua / syncannotations.lua mark per-book sync timestamps
on push/pull success
- Rename "Meta Hash" -> "Book Fingerprint" in koplugin and
apps/readest-app SyncInfoDialog.tsx; translations propagated to
all readest-app locales
- "book config" -> "reading progress" wording across user-facing
strings (matches QiuYukang fork terminology)
- Replace "Log out as " / "Login failed: " concat prefixes with
T(_("...%1..."), arg) placeholder pattern (RTL / verb-final friendly)
- pnpm lint:lua: luajit -b syntax check across koplugin .lua files;
soft-skips when luajit is missing locally; CI installs luajit and
runs the check unconditionally
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(rsvp): add persistent context, display settings, and layout improvements, closes#3333
Add always-visible collapsible context panel, font size adjustment,
ORP color selection, and stable context window that only rebuilds on
scroll. Refactor speed controls into playback row with collapsible
settings behind a gear icon. Translate new i18n keys across 29 locales.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(selfhost): update docker db schema to match FileRecords (#3527)
In #2636, FileRecord was defined to have updated_at field
which is used when it is accessed from the database. But the
local dev setup is missing this field.
This diff adds an updated_at column to match the expectation
* chore(ci): cache system dependency and rustc outputs in ci
* chore(ci): lint script changed from tsc to tsgo
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Azeem Bande-Ali <me@azeemba.com>
* fix(layout): add top/bottom margin to container in scrolled mode, closes#3463
When showMarginsOnScroll is enabled, the paginator margins are set to 0
but the FoliateViewer container had no compensating padding, causing
header/footer bars to overlap content. Now the container padding aligns
content top to the bottom of the header bar (gridInsets.top + 44px) and
content bottom to the top of the footer bar (52px + safe area padding).
Also fixes the footer bar height constant from 44 to 52 to match
ProgressBar's actual h-[52px].
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore(ci): cache all crates including local path deps in Tauri build
Add cache-all-crates: 'true' to Swatinem/rust-cache so that vendored
local path dependencies (packages/tauri/crates/, packages/tauri-plugins/)
are cached between CI runs instead of being recompiled from scratch.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* chore: bump nodejs version to 24
* fix(sidebar): use position fixed and transform for mobile sidebar
Use position: fixed to prevent horizontal scrolling on the mobile
bottom sheet, and replace style.top with transform: translateY() for
smooth drag performance. Cache element refs to avoid
document.querySelector on every drag frame. Apply the same position:
fixed fix to the notebook panel. Closes#3492
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Huang Xin