fix(reader): turn automatically when highlighting across pages (#4487)

* fix(reader): turn automatically when highlighting across pages (closes #1354)

* refact: Refactor time retrieval to use Date.now()

* fix(reader): rework auto page-turn as a corner-dwell gesture

Rework the initial #1354 implementation into a deliberate corner-dwell
gesture that works across platforms, and fix popup positioning for
cross-page selections.

Trigger:
- While a text selection is active, hold any engagement signal — the
  pointer (web/desktop/iOS), the Android native touchmove, or the
  selection caret — inside a screen corner for 500ms to turn one page:
  bottom-right goes to the next page, top-left to the previous.
- One turn per engagement: a signal must leave the corner and return to
  turn another page, so the user controls it one page at a time.
- The corner is a quarter-ellipse of radius 15% of each axis, measured
  against the reading frame (the <foliate-view> rect) inset by the page
  content margins, so the zone lands on the text — not the margin/footer
  or a sidebar — and the pointer can actually reach it.

Per-platform signals:
- web/desktop/iOS: the iframe pointermove, mapped to window coordinates
  via the iframe element's on-screen rect.
- Android: the selection caret (the only signal during a native handle
  drag, where the handles live in a separate window so their touches
  never reach the Activity) plus a throttled (~10/s) native touchmove
  added in MainActivity.dispatchTouchEvent for content drags.

Android scroll-pin (#873): an active selection pins the container scroll,
which reverted the turn; suspend the pin during the turn and re-anchor it
to the page we land on.

Popup positioning: getPosition decided which selection end was on-screen
using window bounds, so a cross-page selection's off-screen start (which
maps behind the sidebar but inside the window) read "in view" and pinned
the popup off the visible page. Test visibility against the reading frame
instead, and for a multi-page selection anchor to the last on-screen line.

Also: logical view.prev()/next() (RTL-correct); skip in scrolled mode;
pass contentInsets down to the annotator.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Huang Xin <chrox.huang@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
justin-jiajia
2026-06-12 00:11:25 +08:00
committed by GitHub
parent 1a85f251c0
commit d165e8df2c
11 changed files with 634 additions and 9 deletions
@@ -33,12 +33,14 @@
- [Custom fonts disappear on cloud sync (#4410)](custom-fonts-reincarnation-4410.md) — CRDT remove-wins: re-import-after-delete needs a `reincarnation` token or the pull re-applies the tombstone; `addFont`/`addTexture` minted none; fix mirrors dictionary (both cases) + OPDS token style; coverage matrix per kind
## Testing
- [CDP Android WebView profiling](cdp-android-webview-profiling.md) — drive the on-device Readest WebView via adb+CDP to run JS probes/benchmarks inside the live app (no rebuild); gotchas: locked device freezes fetch (not invoke), visible:false throttles setTimeout, `__TAURI_INTERNALS__.convertFileSrc/invoke` always present, books in internal `/data/user/0/...`, fs `read{rid,len}` last-8-bytes=nread, `fs|close` not ACL-allowed, curl mishandles WebView HTTP framing
- [Tauri Rust↔JS parser parity tests](tauri-parser-parity-tests.md) — #4369 native Rust EPUB/MOBI parser; how to cross-check vs foliate-js in the `.tauri.test.ts` WebView suite (CWD disk path for Rust, Vite URL for JS, normalizer-based compare, cover presence-only, desc whitespace-collapse); the `dcterms:modified``published` divergence fix
## Build & Vendoring
- [pdfjs vendor wasm decoders](pdfjs-vendor-wasm-decoders.md) — scanned PDFs blank in CI build only (0.11.2 regression); pdfjs 5.7.x moved JBIG2 to `jbig2.wasm`, `copy-pdfjs-wasm` allow-list dropped it; `cpx` no-errors on empty glob; local stale `public/vendor` (gitignored, not refreshed by `tauri build`) masked it; fix = copy `wasm/*`
## Platform Compat
- [Android NativeFile vs RemoteFile I/O](android-nativefile-remotefile-io.md) — why NativeFile is slow (4-IPC/chunk + bridge serialization, tauri#9190); RemoteFile CANNOT replace it on Android (asset-protocol Range broken: start>0 → "Failed to fetch", start-0 capped at 1,024,000; plain no-Range fetch returns full file at 281 MB/s); measured 44/100/281 MB/s; speedups = handle-reuse (2.3×), whole-file asset loader (6.3×), or fix wry upstream. Verified live via CDP.
- [Window-state sanitizer (#4398)](window-state-sanitize-4398.md) — Windows launch crash (WebView2 0x80070057) from invalid `.window-state.json` (`-32000` minimized sentinel / `0×0`); our plugin already has upstream #253 fix so bad files are stale; defense-in-depth `window-state-sanitizer` plugin registered BEFORE window-state (plugin init = registration order); coord threshold `-16000` (~halfway to the -32000 sentinel; real desktops sit a few thousand px off origin) keeps multi-monitor negatives
## Feature Notes
@@ -0,0 +1,27 @@
---
name: android-nativefile-remotefile-io
description: "Why NativeFile is slow on Android, why RemoteFile (range fetch) can't replace it (asset-protocol Range is broken), measured CDP numbers, and the viable speedups"
metadata:
node_type: memory
type: project
originSessionId: 8057ac9c-2e3e-446d-86aa-29baddfbfe66
---
On-device investigation (Xiaomi 2211133C, Android 16, WebView/Chrome 147, wry 0.54.4) of `src/utils/file.ts` `NativeFile` vs `RemoteFile` Android I/O. Verified live via CDP injection into the running app's WebView.
**Why NativeFile is slow (root cause):** `NativeFile.readData``#readAndCacheChunkSafe` does `open()+seek()+read()+close()` = **4 Tauri IPC round-trips per chunk**, opens a FRESH handle every chunk (never reuses `this.#handle`), and `read()` ships raw bytes across the Android Kotlin↔JS IPC bridge (serialization cost = the unresolved tauri-apps/tauri#9190). Code already notes "~400 ms per IPC round-trip" at `nativeAppService.ts:313`.
**Can RemoteFile replace NativeFile on Android? NO** — and whole-file load is NOT an alternative (RemoteFile's whole point is random access WITHOUT loading the file into RAM). The Tauri/wry Android **asset protocol mishandles Range requests** (still true on WebView 147), which is exactly `RemoteFile.fetchRange`'s mechanism:
- `Range: bytes=START-…` with **START ≥ 1024 → hard `TypeError: Failed to fetch`**; `0 ≤ START < 1024` → body truncated to `1024-START` bytes. Reading the zip central directory (EOF) / OPF / cover = non-zero offsets = all fail. "Known issue" at `nativeAppService.ts:244` — confirmed STILL broken.
- **ROOT CAUSE (localized):** Tauri's `crates/tauri/src/protocol/asset.rs` range logic is CORRECT (seek+read [start,end], 206, Content-Range, Content-Length; `MAX_LEN=1000*1024` cap is BY DESIGN — RemoteFile already chunks at the same `MAX_RANGE_LEN`). The bug is in **wry `src/android/binding.rs`**: it STRIPS the `Content-Length` header ("WebResourceResponse will auto-generate") and hands Android a `ByteArrayInputStream` of the already-sliced partial body + a `Content-Range` header. The Android WebView then **double-applies the offset** (skips another `start` bytes) → `1024-start` truncation, empty body for start≥1024. **Unchanged through wry 0.55.1**, so bumping wry won't fix it; needs an upstream wry patch (or local vendor/patch) and fights Android's intercepted-206 quirks.
- Plain `fetch(assetUrl)` (no Range) returns the full file fast — but loading the whole file defeats RemoteFile's purpose, so NOT a fix.
**Measured (10 MB mobi, 1 MB chunks):** native fresh-handle **44 MB/s** (222 ms) · native one-handle **100 MB/s** (98 ms) · asset plain-fetch **281 MB/s** (35 ms, full file correct). Per-call 4 KB scattered read via NativeFile ≈ **16 ms/op** (kills imports doing many small reads). So: plain-fetch is **6.3×** native and **2.8×** one-handle; just reusing the handle is **2.3×**.
**Per-IPC decomposition (warm):** open 1.33 ms, seek 0.60 ms, read(4 KB) 3.02 ms (read carries ~2.4 ms fixed bridge-serialization beyond the round-trip = the tauri#9190 ceiling), seek+read(1 MB) 8.18 ms.
**SOLUTION (implemented, branch `feat/android-rangefile-protocol`, verified on-device):** a custom `rangefile` URI scheme (`src-tauri/src/range_file.rs`, registered via `register_asynchronous_uri_scheme_protocol`) that carries the byte range in the URL **query** (`http://rangefile.localhost/?path=&start=&end=`) instead of a `Range` header. With NO `Range` header the WebView does no offset re-application and delivers the 200 body verbatim — while bytes still stream through the WebView network stack (not the IPC bridge). Returns 200 + `X-Total-Size` (no `Content-Range`); scope-gated by `asset_protocol_scope().is_allowed()` (same security as asset protocol). TS side: `RemoteFile.fromNativePath(absPath)` (query-range mode, reads `X-Total-Size` on open, `&start=&end=` per fetch, no Range header); wired into `nativeAppService.openFile` Android branch with NativeFile fallback; CSP += `http://rangefile.localhost`.
- **Verified on Xiaomi/Android 16 via CDP:** byte-equal to NativeFile ground truth at ALL offsets (0,1,1024,64K,1M,5M,EOF) — the non-zero starts that failed via asset protocol now work; cache-safe across distinct ranges; real library book opens & renders end-to-end (50 rangefile requests, restored mid-file position). **1.83× faster** small scattered 4KB reads (5.2 vs 9.5 ms); bulk-sequential ≈ par (RemoteFile rarely does whole-file reads; native copyFile fast-path handles those). Why this beat the "200 trick" idea: pre-test showed the WebView re-applies the offset to 200 responses too — it's the *Range request header* that triggers it, so removing the header (range-in-URL) is the actual fix.
- Why this isn't the "single-call IPC command": IPC still pays the tauri#9190 bridge serialization; the rangefile path streams via the network stack. The IPC command (`open+seek+read+close` → 1 IPC, ~2× small reads) remains a valid simpler fallback if the custom scheme ever regresses.
**Side-observation:** installed build's ACL rejects `plugin:fs|close` (allows open/seek/read) → possible `NativeFile` handle-leak / `close()` throw path; verify `fs:default` grants. See [[cdp-android-webview-profiling]].
@@ -0,0 +1,20 @@
---
name: cdp-android-webview-profiling
description: "How to drive the Android WebView via CDP (adb) to run JS probes/benchmarks inside the live Readest app, and the gotchas that waste time"
metadata:
node_type: memory
type: reference
originSessionId: 8057ac9c-2e3e-446d-86aa-29baddfbfe66
---
Driving the on-device Readest WebView via CDP to run JS probes/benchmarks **inside the live app** (no rebuild) — used for the NativeFile/RemoteFile I/O study ([[android-nativefile-remotefile-io]]).
**Setup:** app must be running → `adb shell cat /proc/net/unix | grep webview_devtools_remote_<pid>``adb forward tcp:9222 localabstract:webview_devtools_remote_<PID>`. Discover targets with a Node `http.get` to `/json/list` (set header `Host: localhost`) — **curl mishandles the WebView's HTTP framing and hangs/returns empty**. Connect `ws://127.0.0.1:9222/devtools/page/<id>`, then `Runtime.enable` + `Runtime.evaluate {expression:'(async()=>{...})()', awaitPromise:true, returnByValue:true}`. Helper scripts kept in `/tmp/cdp/` (eval.mjs, disc.mjs).
**Gotchas that burned time:**
- **Locked device freezes `fetch`.** When the screen is locked the page is `visible:false`; Chromium freezes the network task queue so EVERY `fetch()` (same-origin and asset) hangs forever — but Tauri `invoke()` still resolves. Must have the user **unlock + keep Readest foregrounded**. Set `svc power stayon true` + `settings put system screen_off_timeout 1800000` after unlock (revert `stayon false` when done).
- **`visible:false` also throttles `setTimeout`** (background timer coalescing → ~60 s). Don't rely on setTimeout guards in probes when the page may be hidden; `invoke`-only probes still work hidden.
- `window.__TAURI_INTERNALS__` is ALWAYS injected (independent of `withGlobalTauri`) → use `.convertFileSrc(path)` and `.invoke(cmd,args)` from injected JS. Android asset URL = `http://asset.localhost/<encodeURIComponent(path)>`.
- Real book files live in **internal** storage (`/data/user/0/com.bilingify.readest/...`), not the external `Android/data/.../files` dir (that's `forbidden path` to the fs plugin). `$APPCACHE` = `/data/user/0/com.bilingify.readest/cache` holds import temp copies (in asset scope `$APPCACHE/**/*`). `adb run-as` is denied on the release build.
- fs plugin invokes: `plugin:fs|open{path,options}→rid`, `seek{rid,offset,whence}` (Start=0), `read{rid,len}`→ArrayBuffer whose **last 8 bytes are bigendian nread**, `close{rid}` (**not ACL-allowed** in the installed build). `read_dir`/`stat` on out-of-scope abs paths return `forbidden path`. Tauri v2 `BaseDirectory`: AppData=14, AppLocalData=15, AppCache=16.
- zsh: `$PIPESTATUS[0]` is a bash-ism (empty in zsh; use `$pipestatus[1]`) — don't trust it for exit codes.
@@ -29,6 +29,10 @@ class MainActivity : TauriActivity(), KeyDownInterceptor {
private var interceptBackKeyEnabled = false
private var interceptPageTurnerKeysEnabled = false
private var keyLearnModeEnabled = false
// touchmove fires continuously; throttle its dispatch to ~10/s since each one
// is an evaluateJavascript round-trip into the WebView.
private val touchMoveThrottleMs = 100L
private var lastTouchMoveTime = 0L
override fun onWebViewCreate(webView: WebView) {
wv = webView
@@ -85,10 +89,19 @@ class MainActivity : TauriActivity(), KeyDownInterceptor {
MotionEvent.ACTION_CANCEL -> "touchcancel"
MotionEvent.ACTION_POINTER_DOWN -> "touchstart"
MotionEvent.ACTION_POINTER_UP -> "touchend"
MotionEvent.ACTION_MOVE -> "touchmove"
else -> null
}
action?.let { eventType ->
// touchmove fires continuously; throttle its dispatch to ~10/s (each one
// is an evaluateJavascript round-trip). down/up/cancel always go through.
val throttledMove = action == "touchmove" &&
event.eventTime - lastTouchMoveTime < touchMoveThrottleMs
if (action == "touchmove" && !throttledMove) {
lastTouchMoveTime = event.eventTime
}
action?.takeIf { !throttledMove }?.let { eventType ->
val pointerIndex = event.actionIndex
val pointerId = event.getPointerId(pointerIndex)
val x = event.getX(pointerIndex)
@@ -0,0 +1,273 @@
import { afterEach, beforeEach, describe, expect, test, vi } from 'vitest';
import { cleanup, renderHook } from '@testing-library/react';
const DWELL_MS = 500;
// The mocked reading frame is 1000x1000; with the 0.15 quarter-ellipse corner,
// (920,920) sits in the bottom-right and (80,80) in the top-left.
const VW = 1000;
const h = vi.hoisted(() => ({
view: {
next: vi.fn(),
prev: vi.fn(),
goLeft: vi.fn(),
goRight: vi.fn(),
deselect: vi.fn(),
getCFI: vi.fn(() => 'cfi'),
renderer: { containerPosition: 100 },
},
appService: { isAndroidApp: false, isMobile: false },
osPlatform: 'macos',
viewSettings: { scrolled: false } as { scrolled: boolean },
}));
vi.mock('@/context/EnvContext', () => ({
useEnv: () => ({ appService: h.appService }),
}));
vi.mock('@/store/readerStore', () => ({
useReaderStore: () => ({
getView: () => h.view,
getViewSettings: () => h.viewSettings,
getProgress: () => null,
}),
}));
vi.mock('@/store/bookDataStore', () => ({
useBookDataStore: () => ({ getBookData: () => ({}) }),
}));
vi.mock('@/utils/event', () => ({
eventDispatcher: { onSync: vi.fn(), offSync: vi.fn(), on: vi.fn(), off: vi.fn() },
}));
vi.mock('@/app/reader/hooks/useInstantAnnotation', () => ({
useInstantAnnotation: () => ({
isInstantAnnotationEnabled: () => false,
handleInstantAnnotationPointerDown: vi.fn(),
handleInstantAnnotationPointerMove: vi.fn(),
handleInstantAnnotationPointerCancel: vi.fn(),
handleInstantAnnotationPointerUp: vi.fn(),
cancelInstantAnnotation: vi.fn(),
}),
}));
vi.mock('@/utils/misc', async (importActual) => {
const actual = await importActual<typeof import('@/utils/misc')>();
return { ...actual, getOSPlatform: () => h.osPlatform };
});
import { useTextSelector } from '@/app/reader/hooks/useTextSelector';
type Handlers = ReturnType<typeof setup>['result'];
const ZERO_INSETS = { top: 0, right: 0, bottom: 0, left: 0 };
const setup = (contentInsets = ZERO_INSETS) => {
const noop = vi.fn();
return renderHook(() =>
useTextSelector(
'book-1',
contentInsets,
noop,
noop,
noop,
vi.fn(async () => ''),
noop,
),
);
};
// The reading frame (the <foliate-view> in #gridcell-book-1) corners are measured
// against; jsdom doesn't lay out, so its rect is supplied.
let areaRect = { left: 0, top: 0, right: VW, bottom: VW, width: VW, height: VW };
// The book iframe's on-screen offset (negative = scrolled into later columns).
let frameOffset = { left: 0, top: 0 };
// The selection caret rect (in iframe space), for the caret signal.
let caretRect = { left: 0, right: 0, top: 0, bottom: 0 };
let currentSel: Selection | null = null;
const doc = {
getSelection: () => currentSel,
createRange: () => ({
setStart: () => {},
collapse: () => {},
getBoundingClientRect: () => caretRect,
}),
defaultView: { frameElement: { getBoundingClientRect: () => frameOffset } },
} as unknown as Document;
const setSelection = (valid: boolean) => {
const node = document.createTextNode('selected text');
currentSel = {
focusNode: node,
focusOffset: 0,
isCollapsed: !valid,
rangeCount: valid ? 1 : 0,
toString: () => (valid ? 'selected text' : ''),
getRangeAt: () => ({}) as Range,
} as unknown as Selection;
};
// Move the pointer to window point (x, y) while a selection is active.
const pointerMove = (result: Handlers, x: number, y: number, valid = true) => {
setSelection(valid);
result.current.handlePointerMove(doc, 0, { clientX: x, clientY: y } as PointerEvent);
};
// Move the selection caret to window point (x, y) — the other engagement signal.
const caretMove = (result: Handlers, x: number, y: number, valid = true) => {
setSelection(valid);
caretRect = { left: x, right: x, top: y - 5, bottom: y + 5 };
result.current.handleSelectionchange(doc, 0);
};
const advance = () => vi.advanceTimersByTimeAsync(DWELL_MS + 50);
beforeEach(() => {
vi.clearAllMocks();
vi.useFakeTimers();
currentSel = null;
frameOffset = { left: 0, top: 0 };
areaRect = { left: 0, top: 0, right: VW, bottom: VW, width: VW, height: VW };
// The reading frame getReadingAreaRect() queries.
const cell = document.createElement('div');
cell.id = 'gridcell-book-1';
const fv = document.createElement('foliate-view');
fv.getBoundingClientRect = () => areaRect as DOMRect;
cell.appendChild(fv);
document.body.appendChild(cell);
h.appService = { isAndroidApp: false, isMobile: false };
h.osPlatform = 'macos';
h.viewSettings = { scrolled: false };
h.view.renderer.containerPosition = 100;
});
afterEach(() => {
vi.runOnlyPendingTimers();
vi.useRealTimers();
document.getElementById('gridcell-book-1')?.remove();
cleanup();
});
describe('useTextSelector auto page-turn on corner dwell (#1354)', () => {
test('turns to the next page when a signal dwells in the bottom-right corner', async () => {
const { result } = setup();
pointerMove(result, 920, 920);
await advance();
expect(h.view.next).toHaveBeenCalledTimes(1);
expect(h.view.prev).not.toHaveBeenCalled();
});
test('turns to the previous page (not goLeft) in the top-left corner', async () => {
const { result } = setup();
pointerMove(result, 80, 80);
await advance();
expect(h.view.prev).toHaveBeenCalledTimes(1);
expect(h.view.goLeft).not.toHaveBeenCalled();
expect(h.view.next).not.toHaveBeenCalled();
});
test('does not turn when the pointer stays in the center', async () => {
const { result } = setup();
pointerMove(result, 500, 500);
await advance();
expect(h.view.next).not.toHaveBeenCalled();
expect(h.view.prev).not.toHaveBeenCalled();
});
test('does not turn until the dwell has elapsed', async () => {
const { result } = setup();
pointerMove(result, 920, 920);
await vi.advanceTimersByTimeAsync(DWELL_MS - 100);
expect(h.view.next).not.toHaveBeenCalled();
});
test('cancels the turn if the pointer leaves the corner before the dwell', async () => {
const { result } = setup();
pointerMove(result, 920, 920); // arms
pointerMove(result, 500, 500); // leaves the corner -> disengage
await advance();
expect(h.view.next).not.toHaveBeenCalled();
});
test('turns one page per engagement and does not repeat while held', async () => {
const { result } = setup();
pointerMove(result, 920, 920);
await advance();
expect(h.view.next).toHaveBeenCalledTimes(1);
// Still held in the same corner -> no further turns.
pointerMove(result, 920, 920);
await advance();
expect(h.view.next).toHaveBeenCalledTimes(1);
});
test('re-arms after the pointer leaves the corner and returns', async () => {
const { result } = setup();
pointerMove(result, 920, 920);
await advance();
expect(h.view.next).toHaveBeenCalledTimes(1);
pointerMove(result, 500, 500); // leave
pointerMove(result, 920, 920); // return
await advance();
expect(h.view.next).toHaveBeenCalledTimes(2);
});
test('ignores a pointer outside the reading area', async () => {
const { result } = setup();
pointerMove(result, VW + 200, 920); // beyond the frame's right edge
await advance();
expect(h.view.next).not.toHaveBeenCalled();
});
test('does not auto-turn in scrolled mode', async () => {
h.viewSettings = { scrolled: true };
const { result } = setup();
pointerMove(result, 920, 920);
await advance();
expect(h.view.next).not.toHaveBeenCalled();
});
test('does not turn without a valid (non-collapsed) selection', async () => {
const { result } = setup();
pointerMove(result, 920, 920, false);
await advance();
expect(h.view.next).not.toHaveBeenCalled();
});
test('the selection caret is also an engagement signal', async () => {
const { result } = setup();
caretMove(result, 920, 920);
await advance();
expect(h.view.next).toHaveBeenCalledTimes(1);
});
test('measures corners against the content-inset reading area', async () => {
// A 100px inset shrinks the frame to [100,900]: the old outer corner (960,960)
// now falls outside the area, while (860,860) is inside the inset corner.
const { result } = setup({ top: 100, right: 100, bottom: 100, left: 100 });
pointerMove(result, 960, 960);
await advance();
expect(h.view.next).not.toHaveBeenCalled();
pointerMove(result, 860, 860);
await advance();
expect(h.view.next).toHaveBeenCalledTimes(1);
});
test('maps the pointer through the iframe offset (multi-column page)', async () => {
// The iframe is scrolled into a later column: a pointer at clientX=1620 maps
// to window x=920 (1620-700), landing in the bottom-right corner.
frameOffset = { left: -700, top: 0 };
const { result } = setup();
pointerMove(result, 1620, 920);
await advance();
expect(h.view.next).toHaveBeenCalledTimes(1);
});
});
@@ -236,6 +236,27 @@ describe('sel utilities', () => {
expect(result.point).toBeDefined();
expect(result.dir).toBeDefined();
});
it('anchors to the on-screen end when the selection start is off-screen (cross-page)', async () => {
const { getPosition } = await import('@/utils/sel');
// A selection that spans a page boundary: its first line is on the
// previous page (off-screen, negative x); its last line is on the current
// page (visible). The popup must anchor to the visible (last) end.
const mockRange = {
getClientRects: () =>
[
{ top: 400, right: -100, bottom: 420, left: -300 }, // previous page (off-screen)
{ top: 200, right: 300, bottom: 220, left: 100 }, // current page (visible)
] as unknown as DOMRectList,
commonAncestorContainer: document.createElement('div'),
} as unknown as Range;
const rect: Rect = { top: 0, right: 1024, bottom: 768, left: 0 };
const result = getPosition(mockRange, rect, 10);
// 'down' = anchored below the visible last line, not the off-screen start.
expect(result.dir).toBe('down');
expect(result.point.x).toBeGreaterThan(0);
});
});
describe('isPointerInsideSelection', () => {
@@ -216,7 +216,7 @@ const BooksGrid: React.FC<BooksGridProps> = ({ bookKeys, onCloseBook, onGoToLibr
bookKey={bookKey}
isDropdownOpen={dropdownOpenBook === bookKey}
/>
<Annotator bookKey={bookKey} />
<Annotator bookKey={bookKey} contentInsets={contentInsets} />
<SearchResultsNav bookKey={bookKey} gridInsets={gridInsets} />
<BooknotesNav bookKey={bookKey} gridInsets={gridInsets} toc={bookDoc.toc || []} />
<FootnotePopup bookKey={bookKey} bookDoc={bookDoc} />
@@ -42,6 +42,7 @@ import {
flushDeferredAction,
runOrDeferAction,
} from '../../utils/deferredAction';
import { Insets } from '@/types/misc';
import { runSimpleCC } from '@/utils/simplecc';
import { getWordCount } from '@/utils/word';
import { getIndexFromCfi, isCfiInLocation } from '@/utils/cfi';
@@ -80,7 +81,10 @@ import {
mergeImportedBookNotes,
} from '@/services/annotation/providers/mrexpt';
const Annotator: React.FC<{ bookKey: string }> = ({ bookKey }) => {
const Annotator: React.FC<{ bookKey: string; contentInsets: Insets }> = ({
bookKey,
contentInsets,
}) => {
const _ = useTranslation();
const { envConfig, appService } = useEnv();
const { settings, setSettingsDialogBookKey, setSettingsDialogOpen, setActiveSettingsItemId } =
@@ -273,6 +277,7 @@ const Annotator: React.FC<{ bookKey: string }> = ({ bookKey }) => {
handleTouchEnd,
handlePointerDown,
handlePointerMove,
handleNativeTouchMove,
handlePointerCancel,
handlePointerUp,
handleSelectionchange,
@@ -281,6 +286,7 @@ const Annotator: React.FC<{ bookKey: string }> = ({ bookKey }) => {
handleContextmenu,
} = useTextSelector(
bookKey,
contentInsets,
setSelection,
setEditingAnnotation,
setExternalDragPoint,
@@ -314,6 +320,9 @@ const Annotator: React.FC<{ bookKey: string }> = ({ bookKey }) => {
androidTouchEndRef.current = false;
cancelDeferredAction(deferredQuickActionRef.current);
handleTouchStart();
} else if (ev.type === 'touchmove') {
// The Android pointer engagement signal (throttled in MainActivity.kt).
handleNativeTouchMove(ev.x, ev.y, doc);
} else if (ev.type === 'touchend') {
androidTouchEndRef.current = true;
handleTouchEnd();
@@ -1,5 +1,6 @@
import { useEffect, useRef } from 'react';
import { BookNote } from '@/types/book';
import { Insets } from '@/types/misc';
import { useEnv } from '@/context/EnvContext';
import { useReaderStore } from '@/store/readerStore';
import { useBookDataStore } from '@/store/bookDataStore';
@@ -8,8 +9,97 @@ import { eventDispatcher } from '@/utils/event';
import { isPointerInsideSelection, Point, TextSelection } from '@/utils/sel';
import { useInstantAnnotation } from './useInstantAnnotation';
const ZERO_INSETS: Insets = { top: 0, right: 0, bottom: 0, left: 0 };
// The selection focus must rest in a screen corner for this long before the
// page auto-turns, so merely passing a corner mid-drag doesn't flip the page.
const AUTO_TURN_DWELL_MS = 500;
// The corner zone is a quarter-ellipse within this radius of the actual corner
// (as a fraction of each axis). Kept tight so it is only the corner itself — a
// larger/rectangular zone catches normal selections that end in the lower-right
// of the page and turns the page unexpectedly.
const AUTO_TURN_CORNER_FRACTION = 0.15;
type Corner = 'br' | 'tl';
// Which screen corner a point sits in: bottom-right turns forward, top-left
// turns back. The zone is a quarter-ellipse of radius FRACTION around each
// corner. Returns null when the point is in neither.
const cornerOf = (x: number, y: number, w: number, h: number): Corner | null => {
if (w <= 0 || h <= 0) return null;
const rx = w * AUTO_TURN_CORNER_FRACTION;
const ry = h * AUTO_TURN_CORNER_FRACTION;
const inEllipse = (dx: number, dy: number) => (dx / rx) ** 2 + (dy / ry) ** 2 <= 1;
if (inEllipse(w - x, h - y)) return 'br';
if (inEllipse(x, y)) return 'tl';
return null;
};
// Map a window-coordinate point to the corner of the reading area it sits in,
// if any. Corners are measured against `area` (the visible text bounds in window
// coordinates) so they land on the text, not the page margins or a sidebar.
const cornerAt = (xWin: number, yWin: number, area: DOMRect | null): Corner | null => {
if (!area || area.width <= 0 || area.height <= 0) return null;
const x = xWin - area.left;
const y = yWin - area.top;
// Ignore a point outside the visible text (e.g. the selection caret jumping
// into the next, off-screen column while dragging at the edge).
if (x < 0 || x > area.width || y < 0 || y > area.height) return null;
return cornerOf(x, y, area.width, area.height);
};
// Window-coordinate position of the selection focus (caret), or null. The book
// content lives in a (possibly very wide, multi-column) iframe translated by the
// pagination offset, so map the caret from iframe space via the iframe element's
// on-screen rect.
const focusCaretWindowPos = (doc: Document, sel: Selection): { x: number; y: number } | null => {
const focusNode = sel.focusNode;
const win = doc.defaultView;
if (!focusNode || !win) return null;
let rect: DOMRect;
try {
const range = doc.createRange();
const offset =
focusNode.nodeType === Node.TEXT_NODE
? Math.min(sel.focusOffset, (focusNode.textContent ?? '').length)
: sel.focusOffset;
range.setStart(focusNode, offset);
range.collapse(true);
rect = range.getBoundingClientRect();
} catch {
return null;
}
// An unmeasurable range (e.g. focus on an empty element) collapses to 0,0,0,0.
if (rect.top === 0 && rect.bottom === 0 && rect.left === 0 && rect.right === 0) return null;
const feRect = win.frameElement?.getBoundingClientRect();
return {
x: (rect.left + rect.right) / 2 + (feRect?.left ?? 0),
y: (rect.top + rect.bottom) / 2 + (feRect?.top ?? 0),
};
};
// The reading frame in window coordinates: the <foliate-view> element's rect
// (a stable element, so it has a sensible page-sized width — unlike the visible
// text range, whose box spans the whole multi-column iframe), inset by the page
// content margins so the corner zone lands on the text area, not the margin.
// Falls back to the reading container (gridcell).
const getReadingAreaRect = (bookKey: string, insets: Insets = ZERO_INSETS): DOMRect | null => {
const cell = document.querySelector(`#gridcell-${bookKey}`);
if (!cell) return null;
const frame = cell.querySelector('foliate-view') ?? cell;
const r = frame.getBoundingClientRect();
if (r.width <= 0 || r.height <= 0) return null;
return new DOMRect(
r.left + insets.left,
r.top + insets.top,
Math.max(0, r.width - insets.left - insets.right),
Math.max(0, r.height - insets.top - insets.bottom),
);
};
export const useTextSelector = (
bookKey: string,
contentInsets: Insets,
setSelection: React.Dispatch<React.SetStateAction<TextSelection | null>>,
setEditingAnnotation: React.Dispatch<React.SetStateAction<BookNote | null>>,
setExternalDragPoint: React.Dispatch<React.SetStateAction<Point | null>>,
@@ -23,6 +113,10 @@ export const useTextSelector = (
const bookData = getBookData(bookKey);
const osPlatform = getOSPlatform();
// The reading frame inset by the page content margins, used to measure the
// auto-turn corners so they land on the text area, not the margin.
const readingAreaRect = (): DOMRect | null => getReadingAreaRect(bookKey, contentInsets);
const isPopuped = useRef(false);
const isUpToPopup = useRef(false);
const isTextSelected = useRef(false);
@@ -32,6 +126,15 @@ export const useTextSelector = (
const isInstantAnnotating = useRef(false);
const isInstantAnnotated = useRef(false);
const annotationStartPoint = useRef<Point | null>(null);
const autoTurnTimer = useRef<ReturnType<typeof setTimeout> | null>(null);
// The corner an input signal is currently engaged in. Stays set after a turn so
// the dwell can't re-arm while held — a signal must leave the corner and return
// to turn another page.
const engagedCorner = useRef<Corner | null>(null);
const isAutoTurning = useRef(false);
// Latest pointer position in window coords (from pointermove or, on Android,
// native touchmove). One of the engagement signals alongside the caret.
const pointerPos = useRef<{ x: number; y: number } | null>(null);
const {
isInstantAnnotationEnabled,
@@ -98,6 +201,14 @@ export const useTextSelector = (
};
const handlePointerMove = (doc: Document, index: number, ev: PointerEvent) => {
// The listener lives on the book iframe's document, so ev.clientX/Y are in
// the (very wide, multi-column) iframe viewport. Map to window coordinates
// via the iframe element's on-screen rect, like the selection caret.
const feRect = doc.defaultView?.frameElement?.getBoundingClientRect();
pointerPos.current = {
x: ev.clientX + (feRect?.left ?? 0),
y: ev.clientY + (feRect?.top ?? 0),
};
if (isInstantAnnotating.current) {
// In scroll mode, detect gesture direction before committing to annotation.
// Cancel if the gesture is along the scroll axis (vertical for normal, horizontal
@@ -116,10 +227,50 @@ export const useTextSelector = (
}
ev.preventDefault();
isInstantAnnotated.current = handleInstantAnnotationPointerMove(doc, index, ev);
return;
}
// Pointer-driven auto page-turn (#1354) for web/desktop/iOS, where the
// pointer is the reliable, stable signal at the corner. Android uses the
// caret in handleSelectionchange (no pointermove during a selection drag).
// Android uses native touchmove (handleNativeTouchMove) — the iframe
// pointermove doesn't fire there during a native selection drag.
const isAndroid = osPlatform === 'android' && appService?.isAndroidApp;
if (isAndroid) return;
const viewSettings = getViewSettings(bookKey);
const sel = doc.getSelection();
const valid = !!sel && isValidSelection(sel);
const corner = !viewSettings?.scrolled && valid ? pointerCornerNow() : null;
noteCorner(corner, doc);
};
// Android native touchmove — the pointer engagement signal during a native
// selection drag (the iframe pointermove doesn't fire there). The native x/y
// are physical device pixels relative to the window; convert to CSS px.
const handleNativeTouchMove = (x: number, y: number, doc: Document) => {
const dpr = window.devicePixelRatio || 1;
pointerPos.current = { x: x / dpr, y: y / dpr };
const viewSettings = getViewSettings(bookKey);
const sel = doc.getSelection();
const valid = !!sel && isValidSelection(sel);
const corner = !viewSettings?.scrolled && valid ? pointerCornerNow() : null;
noteCorner(corner, doc);
};
// Disengage and drop any pending corner page-turn (e.g. when the selection is
// cleared).
const cancelAutoTurn = () => {
engagedCorner.current = null;
if (autoTurnTimer.current) {
clearTimeout(autoTurnTimer.current);
autoTurnTimer.current = null;
}
};
const handlePointerCancel = (_doc: Document, _index: number, ev: PointerEvent) => {
// NB: don't cancel the auto-turn here — on Android pointercancel fires mid
// edge-drag (browser takes over for scrolling), which is exactly when the
// user is dragging into the corner. Cancel only on a real release.
if (isInstantAnnotating.current) {
stopInstantAnnotating(ev);
handleInstantAnnotationPointerCancel();
@@ -177,6 +328,68 @@ export const useTextSelector = (
const handleTouchEnd = () => {
isTouchStarted.current = false;
};
// The corner the latest pointer (pointermove / native touchmove) position is in.
const pointerCornerNow = (): Corner | null => {
const p = pointerPos.current;
return p ? cornerAt(p.x, p.y, readingAreaRect()) : null;
};
// The corner the selection caret (focus) is in.
const caretCornerNow = (doc: Document): Corner | null => {
const sel = doc.getSelection();
if (!sel || !isValidSelection(sel)) return null;
const pos = focusCaretWindowPos(doc, sel);
return pos ? cornerAt(pos.x, pos.y, readingAreaRect()) : null;
};
// Whether any input signal (pointer/touch or caret) is currently in corner `c`.
const inCorner = (c: Corner, doc: Document): boolean =>
pointerCornerNow() === c || caretCornerNow(doc) === c;
// Once a signal has stayed inside a corner for AUTO_TURN_DWELL_MS, turn one page
// (#1354). One turn per engagement — a signal must leave the corner and return
// to turn again (engagedCorner stays set after a turn so the dwell can't re-arm
// while held).
const armDwell = (corner: Corner, doc: Document) => {
if (autoTurnTimer.current) return;
autoTurnTimer.current = setTimeout(() => {
autoTurnTimer.current = null;
const sel = doc.getSelection();
// Skip if the selection ended or every signal left the corner during the dwell.
if (isAutoTurning.current || !sel || !isValidSelection(sel) || !inCorner(corner, doc)) return;
// On Android an active selection pins the container scroll (issue #873 in
// handleScroll). A deliberate page-turn IS a container scroll, so it gets
// snapped straight back unless we suspend the pin for the turn and then
// re-anchor it to the page we land on.
isAutoTurning.current = true;
// Logical next()/prev() so RTL books turn the correct way.
const turning = corner === 'br' ? view?.next() : view?.prev();
Promise.resolve(turning).finally(() => {
selectionPosition.current = view?.renderer?.containerPosition ?? selectionPosition.current;
isAutoTurning.current = false;
});
}, AUTO_TURN_DWELL_MS);
};
// Feed a corner detected from an input signal (pointer/touch/caret) into the
// dwell state machine. Entering a corner arms the dwell; once no signal is in
// the engaged corner any more it disengages so a re-entry can turn again.
const noteCorner = (corner: Corner | null, doc: Document) => {
if (isAutoTurning.current) return;
if (corner) {
if (engagedCorner.current !== corner) {
engagedCorner.current = corner;
armDwell(corner, doc);
}
} else if (engagedCorner.current && !inCorner(engagedCorner.current, doc)) {
engagedCorner.current = null;
if (autoTurnTimer.current) {
clearTimeout(autoTurnTimer.current);
autoTurnTimer.current = null;
}
}
};
const handleSelectionchange = (doc: Document, index: number) => {
// Available on iOS, Android and Desktop, fired when the selection is changed.
// On Android native app, this is the primary way to detect text selection.
@@ -185,8 +398,20 @@ export const useTextSelector = (
// selectionchange for touch/pen input to pick up native text selections.
const isAndroid = osPlatform === 'android' && appService?.isAndroidApp;
const isTouchInput = lastPointerType.current === 'touch' || lastPointerType.current === 'pen';
const sel = doc.getSelection() as Selection;
const viewSettings = getViewSettings(bookKey);
// Auto page-turn (#1354): the selection caret is one of the engagement
// signals on every platform (and the only one on Android during a native
// selection drag, where pointer/touch-move don't fire). Feed it into the same
// dwell machine the pointer uses.
if (isValidSelection(sel)) {
noteCorner(!viewSettings?.scrolled ? caretCornerNow(doc) : null, doc);
} else {
cancelAutoTurn();
}
if (!isAndroid && !isTouchInput) return;
if (isValidSelection(sel)) {
// On desktop with mouse, defer to pointerup for valid selections.
if (!isAndroid && !isTouchInput) return;
@@ -211,12 +436,16 @@ export const useTextSelector = (
const handleScroll = () => {
// Prevent the container from scrolling when text is selected in paginated mode
// FIXME: this is a workaround for issue #873
// TODO: support text selection across pages
if (osPlatform !== 'android' || !appService?.isAndroidApp) return;
const viewSettings = getViewSettings(bookKey);
if (viewSettings?.scrolled) return;
// Don't fight a deliberate auto page-turn (#1354): without this the pin
// below snaps the container straight back to the selection-start page and
// the turn never sticks (the Android-only failure mode).
if (isAutoTurning.current) return;
if (isTextSelected.current && view?.renderer && selectionPosition.current !== null) {
view.renderer.containerPosition = selectionPosition.current;
}
@@ -270,6 +499,7 @@ export const useTextSelector = (
eventDispatcher.onSync('iframe-single-click', handleSingleClick);
return () => {
eventDispatcher.offSync('iframe-single-click', handleSingleClick);
if (autoTurnTimer.current) clearTimeout(autoTurnTimer.current);
};
// eslint-disable-next-line react-hooks/exhaustive-deps
}, []);
@@ -283,6 +513,7 @@ export const useTextSelector = (
handleTouchEnd,
handlePointerDown,
handlePointerMove,
handleNativeTouchMove,
handlePointerCancel,
handlePointerUp,
handleSelectionchange,
+1 -1
View File
@@ -41,7 +41,7 @@ export type FileInfo = {
};
export type NativeTouchEventType = {
type: 'touchstart' | 'touchcancel' | 'touchend';
type: 'touchstart' | 'touchmove' | 'touchcancel' | 'touchend';
pointerId: number;
x: number;
y: number;
+32 -3
View File
@@ -292,9 +292,38 @@ export const getPosition = (
),
dir: 'down',
} as Position;
const startInView = pointIsInView(start.point);
const endInView = pointIsInView(end.point);
if (!startInView && !endInView) return { point: { x: 0, y: 0 } };
// Decide which selection end is on-screen by testing its UNCLAMPED line-rect
// midpoint against the READING FRAME (`rect`) — not the window. A cross-page
// selection's off-screen start maps to a negative/sidebar x that is still
// inside the window, so a window-based check would wrongly read it "in view"
// and pin the popup off the visible page (#1354).
const midX = (r: Rect) => (r.left + r.right) / 2;
const midY = (r: Rect) => (r.top + r.bottom) / 2;
const inFrame = (px: number, py: number) =>
px > rect.left && px < rect.right && py > rect.top && py < rect.bottom;
const startInView = inFrame(midX(first), midY(first));
const endInView = inFrame(midX(last), midY(last));
if (!startInView && !endInView) {
// Multi-page selection: both ends are off the visible page, but the middle
// may cross it. Anchor to the last on-screen line so the popup tracks the
// visible part of the selection.
const v = rects
.map((r) => frameRect(frame, r, sx, sy))
.filter((r) => inFrame(midX(r), midY(r)))
.at(-1);
if (v) {
return {
point: constrainPointWithinRect(
{ x: midX(v) - rect.left, y: v.bottom - rect.top + 6 },
rect,
paddingPx,
),
dir: 'down',
} as Position;
}
// Otherwise fall through and anchor to an end so the popup still shows
// (the constrained points are always within the frame, never {0,0}).
}
if (!startInView) return end;
if (!endInView) return start;
return start.point.y > window.innerHeight - end.point.y ? start : end;