diff --git a/apps/readest-app/.claude/memory/MEMORY.md b/apps/readest-app/.claude/memory/MEMORY.md index b7d5cc59..72b2c5d9 100644 --- a/apps/readest-app/.claude/memory/MEMORY.md +++ b/apps/readest-app/.claude/memory/MEMORY.md @@ -43,7 +43,10 @@ - [TTS sync paragraph+RSVP (#3235, PR #4576)](tts-sync-paragraph-rsvp-3235.md) β€” TTS-is-clock follow: canonical `tts-position{cfi,kind:word|sentence,sectionIndex,sequence}`; in-mode πŸ”Š audio toggle (`build{Paragraph,Rsvp}TtsSpeakDetail`, live-range gate); **current word/sentence highlight painted on the overlay CLONE via CSS Custom Highlight API** (no DOM mutation, spans inline; offsets relative to para-start map 1:1 to clone, `getTextSubRange` reuse, index-tagged vs stale); kind-gating `decideParagraphTtsHighlight` (Edge word wins, skip coarse sentence); `::highlight()` from `ttsHighlightOptions` ## Build & Vendoring +- [Turbopack build-cache OOM + gated Docker standalone (#4619)](turbopack-build-cache-oom-docker-standalone.md) β€” interrupted-build partial `turbopackFileSystemCacheForBuild` cache β†’ 42 workers/18GB-swap freeze (clean build=~6.5GB); disabled the flag; `output:'standalone'` gated on `BUILD_STANDALONE` (Docker-only); tauri CI uses `next dev` (config-independent) +- [Deps/security override workflow](deps-security-overrides-workflow.md) β€” fix transitive npm Dependabot alerts: main monorepo overrides live in `pnpm-workspace.yaml` (NOT root package.json); `packages/tauri-plugins` is a SEPARATE submodule project w/ own lockfile + `minimumReleaseAge` (main workspace has no age gate); bound 0.x overrides like `vite`; verify via test+lint+build-web. PR #4618 (esbuild 0.28.1, vitest 4.1.9) - [R2 rclone CreateBucket 403 (#4588)](r2-rclone-createbucket-403.md) β€” single-file `rclone copyto`/`moveto` probes CreateBucket β†’ 403 on object-scoped R2 token; use a directory `rclone copy` (or `no_check_bucket=true`); broke nightly assemble, not the release flow +- [Deploy workers.dev SNI-block + proxy](deploy-workers-dev-sni-proxy.md) β€” pnpm deploy crash (CN): workers.dev SNI-blocked (DoH useless), R2 populate WS hangs even via proxy; shipped fix = `dangerous.disableIncrementalCache:true` in open-next.config (stock deploy skips populate; readest has no ISR so runtime no-op) - [pdfjs vendor wasm decoders](pdfjs-vendor-wasm-decoders.md) β€” scanned PDFs blank in CI build only (0.11.2 regression); pdfjs 5.7.x moved JBIG2 to `jbig2.wasm`, `copy-pdfjs-wasm` allow-list dropped it; `cpx` no-errors on empty glob; local stale `public/vendor` (gitignored, not refreshed by `tauri build`) masked it; fix = copy `wasm/*` ## Platform Compat diff --git a/apps/readest-app/.claude/memory/deploy-workers-dev-sni-proxy.md b/apps/readest-app/.claude/memory/deploy-workers-dev-sni-proxy.md new file mode 100644 index 00000000..16ea4b09 --- /dev/null +++ b/apps/readest-app/.claude/memory/deploy-workers-dev-sni-proxy.md @@ -0,0 +1,24 @@ +--- +name: deploy-workers-dev-sni-proxy +description: "pnpm deploy crashes in China β€” workers.dev SNI-blocked, wrangler ws WebSocket bypasses http_proxy; fix = NODE_OPTIONS preload" +metadata: + node_type: memory + type: project + originSessionId: 65342d98-7939-41ed-9e10-2efc466946b1 +--- + +`pnpm deploy` (and `pnpm upload`) crashed for chrox (behind GFW, Privoxy at `http://127.0.0.1:8118`) with an **unhandled `ws` `'error'` event β†’ Node process crash** (ETIMEDOUT to Facebook/Vultr/Twitter IPs). + +**Trigger:** `opennextjs-cloudflare deploy`/`upload` ALWAYS runs `populateCache({target:"remote"})` BEFORE the real deploy (no skip flag; only `cacheChunkSize`/`env` knobs). That step calls wrangler's `unstable_startWorker({remote:true})`, which opens a **WebSocket** to a `*.workers.dev` edge host. (`preview` uses `target:"local"` β†’ unaffected.) + +**Root cause (two layers):** +1. `*.workers.dev` is **SNI-blocked** by the GFW, not merely DNS-poisoned. Proof: encrypted DoH gives the REAL Cloudflare IPs (104.18.x), but a *direct* TLS connect to that correct IP with SNI=workers.dev is still `Connection reset by peer` before TLS starts. So **DoH/dnscrypt-proxy does NOT help** β€” the connection must avoid being made directly at all. +2. wrangler's REST calls honor `http_proxy` (undici `ProxyAgent`/`EnvHttpProxyAgent`), but the raw `ws` handshake falls back to `https.globalAgent` and **ignores proxy env**. So it connects directly β†’ SNI reset β†’ crash. The crash fires async (unhandled WS 'error'), so `opennextjs-cloudflare deploy`'s `await` can't catch it. + +**Attempt 1 β€” proxy preload (tried, then REMOVED):** a zero-dep preload that replaced `https.globalAgent` with a `CONNECT`-tunnel agent (CONNECT hides the SNI = defeats the block + does remote DNS; loopback bypassed so the local populate worker on 127.0.0.1 still works). Verified `https.get('https://workers.dev')` β†’ 301 via proxy. This got PAST the WebSocket crash β€” the local populate worker started and enumerated all 17 cache assets β€” **BUT the actual R2 writes through the remote binding then timed out** ("Failed to send request to R2 worker: aborted due to timeout", retrying forever). The proxy establishes the connection but can't reliably carry the sustained cache-write traffic. So the preload alone is NOT sufficient. Deleted it. + +**Attempt 2 β€” replicate `wrangler deploy` in the npm script (tried, then reverted):** skip populate by bypassing `opennextjs-cloudflare deploy` and running `CLOUDFLARE_LOAD_DEV_VARS_FROM_DOT_ENV=false OPEN_NEXT_DEPLOY=true wrangler deploy` directly (traced from `runWrangler`: stock deploy's real step is plain `wrangler deploy` vs `wrangler.toml` which has `main=.open-next/worker.js`+all bindings; no generated config/skew mapping; the env flag stops wrangler 4.x auto-loading `.env`/`.dev.vars` into the worker β€” OpenNext's adapter handles env). Works, but hacky (replicates internals, drift risk). + +**Fix that SHIPPED β€” config flag (cleanest).** populateCache is gated by `if (!config.dangerous?.disableIncrementalCache && incrementalCache)`. So in `open-next.config.ts`: `config.dangerous = { ...config.dangerous, disableIncrementalCache: true }`. This makes the STOCK `opennextjs-cloudflare deploy`/`upload` skip populate (no script hack, no env flag, no drift) β€” reverted package.json to stock. Caveat: it's the SAME flag the runtime reads, so it ALSO disables the runtime incremental cache β€” **but readest uses ZERO ISR (no `revalidate`/`unstable_cache`/`'use cache'`/`generateStaticParams`), so runtime caching is a no-op anyway β†’ no real loss.** Re-enable = delete the one line (from a network that can reach the CF edge). `defineCloudflareConfig` returns `OpenNextConfig` (broad type; `dangerous.disableIncrementalCache?: boolean` exists), tsgo+biome clean. `preview` was always fine (local populate). + +Related: [[turbopack-build-cache-oom-docker-standalone]], [[r2-rclone-createbucket-403]]. diff --git a/apps/readest-app/.claude/memory/deps-security-overrides-workflow.md b/apps/readest-app/.claude/memory/deps-security-overrides-workflow.md new file mode 100644 index 00000000..65d22926 --- /dev/null +++ b/apps/readest-app/.claude/memory/deps-security-overrides-workflow.md @@ -0,0 +1,45 @@ +--- +name: deps-security-overrides-workflow +description: "How to fix transitive npm Dependabot alerts in the readest monorepo (pnpm-workspace overrides, where config lives, tauri-plugins is separate)" +metadata: + node_type: memory + type: reference + originSessionId: c61e7dd2-4033-4bd1-8f32-22056e4ef322 +--- + +Fixing transitive npm Dependabot security alerts (manifest `pnpm-lock.yaml`). + +**Where pnpm config lives (non-obvious):** the MAIN monorepo's `overrides`, +`patchedDependencies`, `onlyBuiltDependencies`, `allowBuilds` are in +**`pnpm-workspace.yaml`** (newer pnpm style) β€” NOT the root `package.json` +(root `package.json` has no `pnpm` section). The root `pnpm-lock.yaml` is what +Dependabot scans; alerts report manifest `pnpm-lock.yaml` = this root lockfile. + +**`packages/tauri-plugins` is a SEPARATE project**, not part of the main pnpm +workspace. It's a git submodule (`tauri-plugins-workspace`) with its OWN +`pnpm-lock.yaml` and its own `package.json` `pnpm.overrides` + +`minimumReleaseAge: 4320`. The `minimumReleaseAge` (3-day age gate) applies ONLY +there β€” the main monorepo has NO age gate, so `^X` specs resolve to the very +latest matching version. Dependabot does not scan the tauri-plugins lockfile. +`pnpm-workspace.yaml` `packages:` = `apps/*`, send-email worker, extensions, +`packages/foliate-js` (NOT tauri-plugins). + +**Recipe for a transitive advisory:** +1. Add `pkg: '>=X.Y.Z'` to the `overrides:` block in `pnpm-workspace.yaml` + (forces all transitive instances up). For risky 0.x packages, BOUND it like + the existing `vite: '>=7.3.2 <8'` (e.g. `esbuild: '>=0.28.1 <0.29'`). +2. For packages that are also DIRECT deps, bump the spec in + `apps/readest-app/package.json` too (e.g. the vitest family: + `vitest`, `@vitest/browser-playwright`, `@vitest/browser-webdriverio`, + `@vitest/coverage-v8` β€” move in lockstep). +3. `pnpm install`, then `grep -oE "pkg@[0-9.]+" pnpm-lock.yaml | sort -u` to + confirm no vulnerable versions remain. +4. Verify: `pnpm test` + `pnpm lint` + `pnpm build-web` (the last exercises + esbuild in the OpenNext/Cloudflare bundle path). + +**Override applicability:** an override forces a transitive version regardless +of the parent's declared range ONLY when the package is a regular dep (no peer +warning). esbuild is a regular dep of vite; vite 7.3.x pins esbuild `^0.27.0` +but esbuild 0.28.x is API-compatible for vite's usage (0.28 changelog = install +integrity + minifier/codegen fixes). Verified via PR #4618 (alerts #238/#239 +esbuildβ†’0.28.1, #240 @vitest/browserβ†’4.1.9). diff --git a/apps/readest-app/.claude/memory/turbopack-build-cache-oom-docker-standalone.md b/apps/readest-app/.claude/memory/turbopack-build-cache-oom-docker-standalone.md new file mode 100644 index 00000000..19d872bb --- /dev/null +++ b/apps/readest-app/.claude/memory/turbopack-build-cache-oom-docker-standalone.md @@ -0,0 +1,20 @@ +--- +name: turbopack-build-cache-oom-docker-standalone +description: "Turbopack build-cache OOM that froze the Mac, and the gated Docker standalone image (PR" +metadata: + node_type: memory + type: project + originSessionId: 41c10f07-ba55-451b-89b4-7e13a5fa3087 +--- + +PR #4619 (merged, squash 6514d4aa5): web Docker production image β†’ Next.js standalone, plus disabling the experimental Turbopack build cache. + +**RAM-meltdown root cause (cost a force-reboot to find).** `experimental.turbopackFileSystemCacheForBuild: true` (beta) is the culprit, NOT the standalone config and NOT the compiler. When a `pnpm build-web` (Turbopack production build) is interrupted mid-compile (e.g. SIGTERM), it leaves a **partial** build cache in `.next/`. The next build on top of that poisoned cache fans out to ~42 worker processes and drives swap to ~18 GB (macOS auto-grows the swapfile) on a 16 GB M1 Pro β†’ thrash β†’ freeze. A *clean* Turbopack `next build` of readest peaks at only ~6.5 GB and fits fine. **Proven by A/B from a cold `rm -rf .next`:** both `output: undefined` and `output: 'standalone'` complete in ~48 s at ~6.5 GB with flat swap; only the warm/partial-cache run exploded. +- Fixes: removed `turbopackFileSystemCacheForBuild` (kept `turbopackFileSystemCacheForDev`); removed the `pull-request.yml` step that cached `.next/cache` for it (now dead). +- Rule: **after interrupting a local build, `rm -rf .next` before retrying.** Never run an unbounded `pnpm build-web` on the dev machine. + +**Watchdog gotcha.** A host-side memory watchdog that polls `ps`/recursive-`pgrep` over the build tree STARVES under thrash (its own syscalls block on paging) and can't fire the kill. Working pattern: sample every 1 s, decision FIRST using only cheap `sysctl vm.swapusage` + `vm_stat`, kill EARLY (avail < 3.8 GB or swap_used > 2.5 GB), `nice -n 19` the build. Even so, prefer Docker (capped VM RAM) or CI over local full builds. + +**Docker standalone, gated.** `output: 'standalone'` + `outputFileTracingRoot` (monorepo root) are gated on a `BUILD_STANDALONE` env flag set ONLY in the `Dockerfile` build stage (`ENV BUILD_STANDALONE=true` before `pnpm build-web`). Every other path keeps original output: Tauri `export`, local `build-web`/dev (`undefined`), and the Cloudflare/OpenNext deploy (`undefined` in config, but OpenNext forces standalone via `setStandaloneBuildMode`β†’`NEXT_PRIVATE_STANDALONE`, so idempotent). `production-stage` copies `.next/standalone` (β†’ `apps/readest-app/server.js` + hoisted `node_modules`) + `.next/static` + `public`, runs non-root `node`, entrypoint `node apps/readest-app/server.js`. `docker-image.yml` builds it on merge only (not PRs), so verify the gate locally. + +**Tauri CI red herring.** `build_tauri_app`'s `run tauri tests` step (`scripts/test-tauri.sh`) uses `next dev` + `tauri dev --features webdriver` β€” both config-output-independent, so next.config `output`/tracing changes can't affect it. Normally 2–3 min (Swatinem/rust-cache restores crates); a cold cache makes it 15–20 min of "Compiling …" that looks stuck but isn't. See [[r2-rclone-createbucket-403]] for other build/deploy CI notes. diff --git a/apps/readest-app/open-next.config.ts b/apps/readest-app/open-next.config.ts index e4bc5eed..c2edd43c 100644 --- a/apps/readest-app/open-next.config.ts +++ b/apps/readest-app/open-next.config.ts @@ -1,6 +1,14 @@ import { defineCloudflareConfig } from '@opennextjs/cloudflare'; import r2IncrementalCache from '@opennextjs/cloudflare/overrides/incremental-cache/r2-incremental-cache'; -export default defineCloudflareConfig({ +const config = defineCloudflareConfig({ incrementalCache: r2IncrementalCache, }); + +// Skip the deploy-time R2 incremental-cache populate step. The app uses no ISR / +// `revalidate` / `unstable_cache`, so the runtime incremental cache is a no-op +// anyway. Remove this line to re-enable (deploy from a network that can reach the +// Cloudflare edge). +config.dangerous = { ...config.dangerous, disableIncrementalCache: true }; + +export default config;